At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, a genetic algorithm evolving the population of LSTM configurations over 5 generations, automatically selecting the number of units, layers, learning rate, window length, and dropout. The average population improves from generation to generation without us guessing. Let's get started. Hello, welcome to episode five of the LSTM series on electrical load forecasting. In the previous episode, we just randomly selected 64 units and 2 layers. There are hundreds of hyperparameter combinations, and trying them all would take too long. The genetic algorithm mimics evolution, good configurations survive and reproduce, while bad ones are eliminated. Today, we're writing it from scratch.
Create a file called ga.py The search space has 5 genes, hidden units 16-128, 1-3 layers, 3 learning rate options, window length 48-336 hours, and dropout 0-0.3. In total, that's over 500 combinations. Chromosomes contain the index of each gene's selection. Tournament selection takes 3 random individuals and selects the best one. Uniform crossover randomly selects each gene from one of the parents, and mutation replaces a gene with a 20% probability. The genetic algorithm function evaluates the entire population, records the best and the average, and then forms a new generation.
Step 2: Elitism & cache fitness
The two best individuals always pass through without change. This is elitism, to ensure the best solution doesn't get lost. The rest are born from the two tournament winners, then mutated. Cace ensures the same configuration is not trained twice, saving a lot of time.
Step 3: Running the genetic algorithm
Create a file called optimasi_ga.py Fitness is validating after training the LSTM with that configuration, with a maximum of 12 epochs and early stopping, and the training window is shifted by 2 hours for faster results. Population of 8 individuals over 5 generations. Run it, the process takes a few minutes, and each trained configuration is printed along with its map. Take note of the results.
Step 4: Evolution results
In the first generation, the random configurations are very diverse. A small model with a short window can give a map above 5%. The next generation becomes more uniform and better, because good genes like large hidden units spread across the entire population. The graph shows the best map and the average population that decreases and converges. The best configuration is saved to best_config.json Done
Done!
We have written a genetic algorithm from scratch with tournament selection, uniform crossover, mutation, elitism, and cache, then used it to search for LSTM architecture and hyperparameters. In the final episode, we evaluate the best model on the 2019 test data and compare it with all other baselines. Currently working on a thesis, dissertation, or research in the field of electrical load forecasting and time series? Cerdas Inovasi Digital helps with deep learning implementation from data load exploration, feature engineering, ARIMA and regression, LSTM with PyTorch, up to hyperparameter optimization with a genetic algorithm, as well as opening deep learning courses with PyTorch. Want to master the entire series? From hourly load data to forecast evaluation? All 6 episodes Complete with a step-by-step PDF guide and source code, available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want a single class only? Buy the single class with lifetime access. Subscribe now on the premium page cerdasinovasidigital.com Contact our WhatsApp, or visit cerdasinovasidigital.com Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
53 video classes1,282 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
