At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, an agent that initially moves randomly in the maze, then learns on its own from rewards and punishments, until it finally finds the shortest path to the exit, like the red line in the graph on the right. No labeled data at all, just experience. Let's get started. Hello, welcome to episode 17 of the Machine Learning with Python series. We are learning the third type of Machine Learning, reinforcement learning. Here there is no labeled data. An agent interacts with the environment, takes an action, then receives a reward in the form of a reward or punishment. The goal is to collect as much reward as possible. This technique is used by Alpago and robots that learn to walk. We will build Q-Learning from scratch.
Step 1: Maze Environment
We create the file labirin.py. First, we build the environment, a 6x6 maze. Number 1 is a wall, 0 is a path. The agent starts at the top-left corner, and the exit is at the bottom-right corner. Each cell is a state, and the agent has 4 actions. Up, down, left, and right.
Step 2: Reward Rules and Random Agent
Then the reward rules, the core of reinforcement learning. The step function receives a state and an action. Then returns the next state, reward, and whether the game is finished. Arriving at the exit gives a reward plus 10. Hitting a wall or going out of bounds gives a punishment of minus 5, and the agent stays in place. Each step normally deducts 0.1, so the agent is encouraged to find the shortest path. We test with an agent moving randomly, it takes hundreds of steps and the reward is very negative.
Step 3: Q-Table and Bellman Equation
Now Q-Learning. The agent stores a Q-Table, which is a table of values for each state-action pair, indicating how good it is to take that action in that state. After each step, the Q value is updated using the Bellman equation: the old Q plus alpha multiplied by the difference between the reward plus gamma multiplied by the best Q in the next state, minus the old Q. Alpha is the learning rate, gamma is how important future rewards are.
Step 4: Epsilon-Greedy and Training
An agent must also balance exploration and exploitation. With the epsilon greedy strategy, the agent sometimes tries random actions to explore, and more often selects the best action according to the Q-Table. Epsilon starts at one, meaning it's always random, then is reduced little by little each episode. We train for 500 episodes and print the progress.
Step 5: Results: Agent Exits the Maze
View terminal. At the beginning, the total reward is very negative and the number of steps is high. After hundreds of episodes, the agent reaches the exit door in just 10 steps, the shortest possible path. The left graph shows the reward per episode increasing and then stabilizing. The right graph shows the maze and the path chosen by the agent after learning, complete with the best action arrow in each box. Done. We have already built the maze environment, reward rules, Q-Table, bellman equation, and epsilon greedy strategy, then trained the agent until it found the exit.
Done!
For much larger problems, the Q-Table is replaced with a neural network, which is called a Deep Q Network. In the final episode, we put everything together, creating a complete machine learning project and deploying it as a web application. Want to master machine learning and data science? Cerdas Inovasi Digital offers Python, machine learning, and AI courses from the basics, prediction model development services, data analysis, and AI application for business or research, as well as selling the complete source code of this tutorial. Want to master the entire series, from machine learning basics to deploying models to the web? All 18 episodes, complete with step-by-step PDF guides, are available to premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want to take just one class? Buy the unit class with lifetime access. Subscribe now on the premium page at cerdasinovasidigital.com Contact our WhatsApp or visit cerdasinovasidigital.com Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
52 video classes1,276 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
