At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, a graph showing how the model's accuracy changes according to the number of selected features, with three feature selection methods. Previously, we also proved that features created with intelligence can multiply the model's capability, without changing the algorithm at all. Let's start. Hello, welcome to episode 12 of the Machine Learning with Python series. There's a famous saying, data and features determine the upper bound of a model, and the algorithm only approaches that bound. In this episode, we learn about feature engineering, which is creating more meaningful features, and feature selection, which is choosing the truly useful features and discarding the rest.
Step 1: Coffee Sales Data
We create a file called fitur.pe. The data is daily sales of a cafe over two years, containing date, temperature, promotional price, and number of visitors. Actual sales are influenced by weekends, certain holiday seasons, and temperature. The problem is, the raw date column cannot be directly understood by the model.
Step 2: Creating New Features
Let's try the basic model with raw features only. Temperature, price, and visitors, using linear regression. The R-squared is low, only around 0.3. Now feature engineering. From the date, we create features for day of the week, weekend indicator, and month. We also create a ratio feature, which is price per visitor, and a temperature squared feature because coffee sales do not increase linearly with temperature. We convert the month using one hot encoding, because month 12 does not mean it is larger than month 1. See the comparison.
Step 3: Before and After Comparison
With the exact same model, R-squared jumps significantly after adding new features. This is the power of feature engineering, incorporating our understanding of the problem into the data.
Step 4: Feature Correlation
Now feature selection. Create a file called seleksi .py. We use the breast cancer dataset which has 30 features. First, look at the correlation between features. The heatmap shows many features that are highly correlated, for example radius, perimeter, and area, which essentially carry the same information.
Step 5: SelectKBest, Mutual Information, FE
There are three feature selection approaches. Filter methods, such as SelectKBest with F-test or mutual information, evaluate each feature individually against the target. Wrapper methods, such as RFI or recursive feature elimination, train the model repeatedly and remove the weakest feature one by one. We look at the top five features according to each method. How many features are actually needed?
Step 6: Optimal Number of Features
We test the number of features from 1 to 30 with cross-validation for all three methods, then plot them. It turns out that with just 10 features, the accuracy is only slightly below that of 30 features, especially with RFI. The model becomes simpler, faster, and easier to explain.
Done!
Done. We have created features from date, ratio, quadratic transformation, and one-hot, then selected features with correlation, SelectKBest, Mutual Information, and RFI. Remember, feature selection should also be done within the pipeline to avoid data leakage. In the next episode, we enter unsupervised learning with clustering for customer segmentation. Want to master machine learning and data science? Cerdas Inovasi Digital offers Python, machine learning, and AI courses from the basics, data model development, data analysis, and AI application services for business and research, as well as selling complete source code of this tutorial. Want to master the entire series, from machine learning introduction to deploying models to the web? All 18 episodes, complete with PDF step-by-step guides, are available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want just one class? Buy a single class with lifetime access. Subscribe now on the premium page cerdasinovasidigital.com. Contact our WhatsApp or visit cerdasinovasidigital.com. Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
52 video classes1,276 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
