At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, a decision tree that can be read by humans, containing simple questions about the chemical content of wine to guess its type. We also build a random forest, a collection of hundreds of trees that vote among themselves, and see which features are the most important. Let's get started. Hello, welcome to episode 7 of the Machine Learning with Python series. This time we are learning about decision trees, a model that works like the game 20 questions. Is the alcohol content above a certain level? If yes, go left, if not, go right. This model is very easy to explain, but it's prone to overfitting. The solution is a random forest that combines many trees into one strong model.
Step 1: Training the Decision Tree
We create a file called tree.py. The data is the built-in wine dataset from scikit-learn, 178 wines from 3 types, with 13 chemical features such as alcohol content, flavonoids, and color intensity. We train the decision tree with a maximum depth of 3 so the tree is easy to read. Then we print the rules in text form using the export text function.
Step 2: Drawing the Decision Tree
Even clearer with an image. The plot tree function draws the entire tree. Each box contains a question, the Gini value, the number of samples, and the majority class. Gini measures how mixed the contents of a box are, 0 means it's pure of one class. The tree always chooses the question that most reduces this mixing. The color of the box indicates the class, the darker the purer.
Step 3: Overfitting and max depth
What happens if the tree is allowed to grow without limits? Create a file called depth.py We try max depth from 1 to 15, then compare training and test accuracy. Training accuracy rises to 100%, but test accuracy peaks at depth 3 and then decreases. A tree that is too deep memorizes the training data, this is overfitting in a decision tree.
Step 4: Random Forest vs Single Tree
Random forest addresses this issue. Create file forest.py The idea is to train hundreds of trees, each with random sample data and random features, then combine their votes. Each tree's error cancels out. We compare a single unlimited tree with a random forest of 200 trees, using 5-fold cross-validation to make the results more reliable. Random forest clearly wins.
Step 5: Feature Importance
A bonus of tree-based models is feature importance. How much each feature contributes to reducing impurity across the entire tree. We sort and display it as a bar chart. Features such as proline, flavonoids, and color intensity are actually the most determining factors for the type of wine.
Done!
Done. We have already learned how decision trees work, how to read the tree and the Gini value, View overfitting through max depth, and use random forest and feature importance. One thing more, tree-based models do not require feature scaling. In the next episode, we learn about support vector machine and kernel tree. Want to master machine learning and data science? Cerdas Inovasi Digital opens Python, machine learning, and AI courses from the basics, Predictive model development, data analysis, and AI application services for business and research as well as selling the complete source code of this tutorial. Want to master the entire series, from machine learning basics to deploying models to the web? All 18 episodes, complete with step-by-step PDF guides, Available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or one-time pay Rp999.000 for lifetime access. Want a single class only? Buy single class with lifetime access. Subscribe now on the premium page cerdasinovasidigital.com Contact our WhatsApp, or visit cerdasinovasidigital.com Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
52 video classes1,276 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
