At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, the accuracy curve of MobileNetV2 is stuck around 75% during feature extraction, then jumps up to 92.7% after fine-tuning, with the learning rate decreasing following cosine annealing. Let's start. Hello, welcome to episode four of the leaf disease detection series using transfer learning and hyperparameter optimization. In the previous episode, we froze the model body, and the accuracy was stuck around 75%, because the ImageNet features don't know about leaf diseases. Today, we are doing the second method of transfer learning, fine-tuning, opening up some top blocks so the features can adapt to tomato leaves.
Step 1: Open the top block
Create a file named vnetune_leaf.py. We create MobileNetV2 and then load the weights from the feature extraction results of the previous episode, so the classification head is already trained. Then, open the last four blocks of the model body. Why the last blocks? The initial CNN blocks recognize general features like edges and colors useful for all images, while the final blocks recognize more specific patterns, and it is this part that needs to be adapted. Now, 1.5 million out of 2.2 million parameters are being trained, 68.8%. The key is the learning rate.
Step 2: Small learning rate & cosine annealing
We use 0.0003, smaller than during feature extraction, so the pre-trained weights are only slightly adjusted, not ruined. Plus, we use cosine annealing LR scheduler, the learning rate decreases following a cosine curve from the initial value down to almost 0 at the end of training. At the beginning, the model learns quickly, at the end, the steps become smoother. The train function from episode 3 we directly reuse with the scheduler parameters.
Step 3: Fine-tuning results
Last two graphs, validation accuracy from feature extraction is connected to fine tuning, and the learning rate at each epoch. Run. Check the terminal, validation accuracy jumps to 80% in the first epoch, then continues to rise up to 92.7% in the sixth epoch. On the left graph, there is a big spike from the blue line to the orange line, and the right graph shows the learning rate decreasing smoothly. A 17% improvement in just 6 epochs. Done.
Done!
We have performed fine tuning on the top layers of MobileNetV2, using a small learning rate and cosine annealing, and increased the validation accuracy from 75.7 to 92.7%. But is 4 layers and a learning rate of 0.0003 the best choice? In the next episode, we will find it automatically with Optuna. Currently working on a thesis, dissertation, or AI research in the field of agricultural AI? Cerdas Inovasi Digital helps implement deep learning, from transfer learning with MobileNet and ResNet, fine tuning, hyperparameter optimization with Optuna, up to web application and result writing, as well as opening deep learning courses with PyTorch. Want to master the entire series, from the Plan Village dataset to the web app for leaf disease detection? All 6 episodes, complete with step-by-step PDF guides and source code, are available to premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want just one class? Buy the single class with lifetime access. Subscribe now on the premium page at cerdasinovasidigital.com. Contact us via WhatsApp, or visit cerdasinovasidigital.com. Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
53 video classes1,287 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
