Project-3 Announcement: Implementing Regression algorithms from Scratch | #100MLProjects #laxmena

Photo by 贝莉儿 DANIST on Unsplash

Success is the sum of small efforts, repeated day-in and day-out. -Robert Collier

About #100MLProjects:
100MLProjects is a Challenge where I try to attain proficiency in Machine Learning and Deep Learning concepts by doing 100 Projects. The complexity of the projects keeps increasing as I progress through the challenge, so other Machine Learning/Deep Learning aspirants can also follow this path.

Note: Thank you, Harshavardini, Keerthivasini, Thameem and Nirmal for validating this project idea and providing additional inputs.

Project #3 Announcement:

Project: Implement Regression Algorithms from Scratch
Goal: To understand how Regression Algorithms are built, and learn the underlying mechanics of the algorithms by building the Algorithms from Scratch.

For this project, I’ll implement the following regression algorithms and compare their performance with the Scikit Learn’s Implementations.

1. Linear Regression
2. Logistic Regression

Advanced Regressions:

1. Huber Regression
2. Ransac Regression
3. Theil-Sen Regression

I’m acquainted with the first two Regression methods — Linear and Logistic, but I have found the later three regressions while checking out Kaggle Notebooks.

Edit(July 02): For the advanced regressions, I’ll use the libraries to build applications of them. I’m planning to implement them from scratch later during the project, as I have to get myself more familiarized with the mathematics behind them.

Motivation:

When I initially wanted to Learn Machine Learning, I wanted a structure or a road map to follow. I reached out to several people, scanned the internet for hours, and found there is just not one definitive standard way to learn Machine Learning. There were different styles, and there is no one style for all.

Initially, I tried the approach, that my colleague(Nirmal) used for himself— “Master Math First, Implementation next”. (Note: He has very good Math skills). I decided to follow his approach.

I purchased the Math books on Linear Algebra, Probability and Statistics, and other important topics he suggested. It was really fascinating to know how these mathematical concepts are used as building blocks to make amazing Machine Learning algorithms. I created a list of all Math concepts that I should have decent expertise on and started learning them from the books. That part was not fun, after a few days I lost interest in the ‘Only Math’ approach. I couldn't find a satisfying measuring metric to track my progress by this approach.

I shared this experience with my college senior, now a Ph.D. student — Harshavardini, where she suggested an alternative approach. The implementation first, and then learn the Math behind it. This way of approaching sounded promising to me, as I had a good programming background, I could easily pick-up things on the implementation part. Also, I really do love building things, the sense of accomplishment after completing it gives you a confidence boost. That boost and energy surge will keep me motivated to do more of these challenges.

This approach perfectly fits me. If you’ve noticed, I’m applying the same approach here for this challenge. In #Project1 and #Project2, I built applications of regression and classification models - The Implementation part. Now, in this third project, I’m unearthing the math behind the screens. Trying to understand the mechanics of the algorithms.

This is going to be an interesting project, I have a strong sense that I’m going to learn a lot from this project. Hoping for the best. I’ll keep documenting my progress along the way.

Suggestions and Feedbacks:

I received suggestions from my friends and well-wishers after having published my previous article on Handwritten Digit recognition.

The article I wrote was mostly walking-through my codebase. With minimal insights on ‘why I did that’ and ‘What challenges I faced during the project’.

Another suggestion was to make a mixed blog article with both tutorial/walk-through and along with the challenges I faced.

So, I will make sure to draft my upcoming articles to give better intuition, answers for ‘why I did that?’ questions, and share the mistakes I made and how I overcame them. Thank you for your feedback and guidance Thameem Abbas and Anirudh.