At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
Here is the final result, one tomato leaf photo has been transformed into nine different variations, cropped, rotated, flipped, and color changed, through a data pipeline that loads all images into memory so that one epoch only takes about one second. Let's get started. Hello, welcome to episode two of the leaf disease detection series using transfer learning and hyperparameter optimization. Our data only has 140 training photos per class, which is too little for deep learning. The solution is augmentation, creating new variations from existing photos every epoch. We also set up an efficient data pipeline, because in the following episodes, the model will be trained multiple times, including dozens of times by Optuna.
Step 1: Training & Evaluation Transformation
Create a file named dataset_underscore_leaf.py. First, define the mean and standard deviation of ImageNet. The pre-trained model we will use was trained with this normalization, so our input must be normalized in the same way. Then two transformation sequences from Torch Vision. Augustus for training data, random resize crop randomly cuts 70-100% of the image, then resizes to 128 pixels, flips horizontally, rotates up to 20 degrees, and color jitter that changes brightness, contrast, and saturation. Eval for validation and testing is not random, resize then crop the center.
Step 2: Dataset in Memory & DataLoader
Next, the leaf data class. In the constructor, all images are opened once and saved in memory, so there is no need to read files from the disk every epoch. Augmentation is still applied in getitem, so every epoch the model sees a new variation. The loader function creates a data loader for the training, validation, or testing part, and saves the dataset in cache. The portion parameter allows using part of the training data, which is useful when optimizing with Optuna later.
Step 3: Viewing Augmentation Results
Now create a file named pipeline_underscore_leaf.py to see the augmentation results. We take one photo of a septoria leaf spot and apply Augustus 9 times. See, each variation is different, with different positions, angles, sizes, and colors, but the disease spot is still visible. This is the principle of augmentation, changing the appearance without changing the label.
Step 4: Measuring Pipeline Speed
Finally, we measure the pipeline speed. The first line is the time to read 1400 files from the disk in one epoch. With memory caching, images are loaded once at the beginning. Then each epoch takes only about one second less, including augmentation, and our training will repeat epochs multiple times. The exact number depends on your computer's speed and your bandwidth, and on large datasets with high-resolution images, the difference can be multiplied. The last batch contains 24 images, because 1400 is not divisible by 32. While the validation batch is shaped as 32 images, 3 channels, 128x128.
Done!
Done. We have created a torch vision augmentation for the down images, image net normalization, dataset that loads images into memory, data loader, and measured the pipeline's speed. In the next episode, we will enter the core of the series, transfer learning with mobile net V2 and resnet 50 through feature extraction. Currently working on a thesis, dissertation, or AI research in the field of agricultural AI?
ye We | cedanostdigolom .
Cerdas Inovasi Digital helps implement deep learning, from transfer learning mobile net and resnet, fine tuning, hyperparameter optimization with optuna, up to web application and result writing, as well as opening deep learning courses with Pythor. Want to master the entire series, from the plan village dataset up to the web application for leaf disease detection? All 6 episodes, complete with PDF step-by-step guides and source code, are available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want a single class only? Buy a single class with lifetime access. Subscribe now at the premium page cerdasinovasidigital.com Contact our WhatsApp, or visit cerdasinovasidigital.com Thank you for watching, see you in the next episode. Thank you for watching,
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
53 video classes1,286 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
