At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, we trained a linear regression model from scratch, without any machine learning library. The left graph shows the error decreasing continuously at each gradient descent step, and the right graph shows the line that fits the data well. This is actually what happens behind the feed command. Let's get started. Hello, welcome to episode three of the machine learning with Python series. Many people are afraid of machine learning mathematics. Take it easy, in this episode we'll learn intuitively and directly prove it with code, vectors and dot product, basic statistics, probability with Bayes' theorem, derivatives as slope and most importantly, gradient descent, the algorithm used by almost all models to learn.
Step 1: Vectors and dot product
We open the terminal and create a vectors.py file. One data row is a vector. For example, a house with an area of 120 square meters, three rooms, and five years old. The linear model calculates predictions using the dot product between features and weights. then add the bias. The dot product is also used to measure the similarity between two vectors, which is called cosine similarity, basics of a recommendation system. Movies A and B, which are both action films, have a high similarity While A and C have low romantic values. Next, statistics.
Step 2: Statistics and normal distribution
This salary data has one extreme value, The boss's salary is 50 million. Notice the mean is pulled to 11.3, while the median remains 7, which better represents the majority. Then we create 10,000 height data points normally distributed, and prove the 68, 95, 99.7 rule. About 68 percent of the data lies within one standard deviation of the mean.
Step 3: Bayes' Theorem
Now the probability with Bayes' theorem, the basis of the knife base algorithm. A disease affects 1 percent of people. The test is accurate 95 percent, but 5 percent of healthy people also test positive. If your test result is positive, what is the probability you are actually sick? It turns out only about 16 percent. We also prove this with a simulation of 1 million people. The lesson is, high accuracy can be misleading on unbalanced data.
Step 4: Derivatives and gradient descent
Enter calculus. Create a file gradient.py. The derivative is the slope of a function. Our function is x squared minus 4x plus 5, and its derivative is 2x minus 4. At the minimum point, the slope is 0. Gradient descent seeks the minimum by stepping in the opposite direction of the slope, repeatedly, like descending a foggy mountain. The size of each step is called the learning rate. Let's try 3 values. Too small, just right, and too big.
Step 5: Effect of learning rate
Look at the graph. Learning rate 0.05 moves very slowly and hasn't reached yet. 0.3 directly reaches the minimum, x equals 2. While 1.02 actually jumps further and further away, or diverges. Learning rate is a very important hyperparameter.
Step 6: Linear regression from scratch
Now the peak part, training linear regression from 0. Create a file called regresi underscore 0.py. We create data y equals 3x plus 7, plus noise. Model starts from W and B0. At each epoch, we calculate the prediction, calculate the error, and loss is the mean squared error. Then compute the derivative of the loss with respect to W and B, and update both of them with gradient descent. We print its progress over several epochs.
Step 7: Training results
See terminal, loss drops from hundreds to below 4, and the final result approaches 3x plus 7. The left graph shows the loss continuously decreasing, The right graph shows the line that fits the data. Congratulations! You just trained a machine learning model without a machine learning library.
N Done!
Done! We have already learned about vectors and dot product, descriptive statistics, normal distribution, base theorem, derivative, and gradient descent. In the next episode, we'll learn about the stage that takes the most time data scientist time, cleaning and exploring data. Want to master machine learning and data science? Cerdas Inovasi Digital opens a Python course, machine learning, and AI from the basics, Prediction model creation service, data analysis, and AI application for business or research, as well as selling the complete source code of this tutorial. Want to master all series of this, from machine learning introduction to deploying models to the web? All 18 episodes, complete with step-by-step PDF guides, are available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or one-time payment of Rp999.000 for lifetime access. Want just one class? Buy single class with lifetime access. Subscribe now at the premium page cerdasinovasidigital.com. Contact our WhatsApp, or visit cerdasinovasidigital.com. Thank you for watching, see you in the next episode. Thank you for watching
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
53 video classes1,279 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
