At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, Optuna automatically tries 14 hyperparameter combinations, stopping unpromising experiments midway, and shows that learning rate is the most influential hyperparameter. The model finally reaches 93% validation accuracy. Let's get started. Hello, welcome to episode five of the leaf disease detection series using transfer learning and hyperparameter optimization. In the previous episode, we selected 4 blocks and a learning rate of 0.0003 based on experience. In research, such choices should be systematically explored. Optuna is a popular hyperparameter optimization library, we just need to write an objective function, and Opt,una will cleverly find the best combination. Create a file named optuna_underscore_daun_dot_py.
Step 1: Objective function & search space
The core of Optuna is the objective function that receives a trial object. Inside it, we define the search space, learning rate from 0.0001 to 0.003 on a logarithmic scale, dropout from 0 to 0.5, number of opened blocks from 0 to 6, and weight decay on a logarithmic scale. The model is built with these values and trained for 3 epochs with half the training data to speed things up.
Step 2: Pruning with callback report
Notice the reporting function. Every epoch, validation accuracy is reported to Optuna with trial report. If the trial's pruned value is true, it means the trial's intermediate result is worse than the median of other trials, so the experiment is stopped earlier with trial pruning. This is pruning, which saves a lot of time. We send this callback to the train function from episode 3. We create a study with the maximize direction
Step 3: Study, TPE sampler, and graphs
TPI sampler, which is a Bayesian algorithm that selects the next combination based on previous results, and median pruner. Optimize runs 14 trials, and each trial is printed in a summary. The best result is saved to JSON, then 2 graphs, trial value history and hyperparameter importance.
Step 4: Running Optuna optimization
Run. The process takes about 2 minutes, the video is sped up. Check the terminal, out of 14 trials, 8 were stopped earlier by the pruner. Trials with 0 open blocks only reached about 60%, confirming the importance of fine-tuning.
Step 4: Optuna optimization
Best trial number 13, with learning rate 0.00021, dropout 0.21, 6 open blocks, and validation accuracy 87.3% in 3 short epochs.
Step 4: Running Optuna optimization
The right graph shows learning rate as the most important hyperparameter, followed by Wade Decai and dropout.
Step 5: Final model yi
Now create the best train underscore file.py. We read the best hyperparameters, Create a model with dropout Load feature extraction weights Open 6 blocks Then train fully for 6 epochs with cosine annealing Run Validation accuracy increases up to 93% Slightly above the manual fine-tuning result of 92.7% The model is saved as the best model.pt
& Done!
Done We have used Optuna with the objective function Search space TPI sampler Median pruner And analysis of hyperparameter importance Then train the final model with the best hyperparameters In the last episode, we evaluate the model on the test data per class Read the confusion matrix And create a web application for leaf disease detection Currently working on a thesis Dissertation Or AI research in the field of agricultural AI Cerdas Inovasi Digital helps implement deep learning From transfer learning with MobileNet and ResNet Fine-tuning Hyperparameter optimization with Optuna until web applications and writing the results, as well as opening deep learning courses with Pythor. Want to master the entire series, from the plan village dataset until the web application for leaf disease detection? All six episodes, complete with step-by-step PDF guides and source code, are available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want just one class? Buy single classes with lifetime access. Subscribe now on the premium page at cerdas inovasi digital.com. Contact our WhatsApp, or visit cerdasinovasidigital.com. Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
52 video classes1,276 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
