All classes 5G Deep RL Resource Allocation Syllabus · Episode 4 of 6

Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward

0 views

Tutorial on formulating MDP for 5G resource block and power allocation as a Gymnasium environment with Python: state, action, reward. Episode 4 of the 5G resource allocation deep reinforcement learning series.

Software: Python (NumPy, PyTorch, Gymnasium, Matplotlib)
Level: Advanced
Duration: ± 11 min
Series total: 1 h 1 min · 6 videos
Published: October 25, 2026

Episode 4 of 6 · view syllabus

Download PDF (6 PDF pages)

0 downloads

Download Project File· 100 KB Unlock All Premium Classes ▶ Watch the video lesson0 plays

▶ Video Lesson

Watch every step above performed live, with English narration.

0 plays

Class book

Page 1 / 6

Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward: Cover: Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward

Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward: Before you start, steps list, and Step 1

Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward: Step 2: Reward & slot transition

Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward: Step 3: Random policy vs PF

Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward: Step 4: Reward & action selection

Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward: Done: Final Result and Further Practice

Cover: Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward

Click a page or use ◀ ▶ to turn · swipe on mobile · to enlarge

Want to go deeper?

Join one of our courses or ask us directly on WhatsApp.

Download PDF· 0 downloads View the full 6-episode syllabus View 5G Resource Allocation Courses Ask on WhatsApp

Recommended classes for you

Picked from the software, topic, and level of “Class 5G Resource Allocation Deep RL #4: MDP Formulation: State, Action, Reward”.

18 episodesFree preview

Python · Beginner–Advanced

Machine Learning Python

18 video lessons with PDF guides, 3 h 41 min in total. Starts with: Introduction to Machine Learning & First Model.

3 h 41 min 61 0

View class

Deep Learning CNN-LSTM Pneumonia Detection

6 episodesFree preview

Python · Intermediate–Advanced

Deep Learning CNN-LSTM Pneumonia Detection

6 video lessons with PDF guides, 57 min in total. Starts with: Chest X-ray Dataset: Loading, Exploration & Data Splitting.

57 min 22 0

View class

EKG Arrhythmia Classification Wavelet Deep Learning

6 episodesFree preview

Python · Intermediate–Advanced

EKG Arrhythmia Classification Wavelet Deep Learning

6 video lessons with PDF guides, 56 min in total. Starts with: EKG & EEG Signals: Characteristics and Public Datasets (MIT-BIH).

56 min 2 0

View class

50 episodesFree preview

Agentic AI · Beginner–Intermediate

Agentic AI Without Coding

50 video lessons with PDF guides, 14 h 29 min in total. Starts with: Chatbots, Automation, and AI Agents: What's the Difference?.

14 h 29 min 66 0

View class

100 episodesFree preview

Claude · Beginner–Advanced

Claude from Zero to Expert

100 video lessons with PDF guides, 15 h 1 min in total. Starts with: What Claude Is and How It Differs from ChatGPT and Gemini.

15 h 1 min 62 0

View class

Reinforcement Learning for Robot Manipulator

6 episodesFree preview

Python · Intermediate–Advanced

Reinforcement Learning for Robot Manipulator

6 video lessons with PDF guides, 49 min in total. Starts with: Forward and Inverse Kinematics of a 2-DOF Robotic Arm.

49 min 1 0

View class

MIMO-OFDM Channel Estimation Deep Learning

6 episodesFree preview

Python · Intermediate–Advanced

MIMO-OFDM Channel Estimation Deep Learning

6 video lessons with PDF guides, 57 min in total. Starts with: OFDM and MIMO Basics: Subcarrier, Pilot, Multipath Channel.

57 min 0 0

View class

6 episodesFree preview

Python · Intermediate–Advanced

Transfer Learning for Leaf Disease

6 video lessons with PDF guides, 1 h 5 min in total. Starts with: PlantVillage Dataset: Folder Structure, Class Distribution & Data Split.

1 h 5 min 9 0

View class

Need more than a video?

Other ways we can help with Python.

Learn with a mentor

Scheduled AI & Machine Learning courses with live guidance and a certificate.

Course

Multi-Objective Optimization Course: NSGA-II, MOPSO & Hybrid Algorithms (Python)

Course

ECG Arrhythmia Classification Course with Wavelet & Deep Learning (PyTorch)

Course

ANFIS + PSO/GA Course for Non-Linear Time Series Forecasting (Python)

See all in AI & Machine Learning →

Have it built for you

No time to learn Python? Our team can do the project for you.

Service

Python Machine Learning — Prediction Models, Customer Segmentation, Fraud Detection & AI Apps

Service

AI Image Creation with Stable Diffusion — Unlimited Custom Visual Production

★ 4.9

Service

AI Image & Video Creation with Leonardo AI — All-in-One Visuals for Business

★ 4.9

See all in AI & Machine Learning →

Get notified about new Python classes

At most one email a week. Unfollow anytime with one click.

Transcript

The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.

Show transcript

Introduction

This is the final result, the 5G resource allocation problem transformed into a standard reinforcement learning environment in gymnasium, with 25 state numbers, 24 action choices, and a reward that balances fairness and energy efficiency. Let's start. Hello, welcome to episode four of the 5G resource allocation series with deep reinforcement learning. Reinforcement learning requires a problem in the form of a Markov decision process. The agent observes the state, selects an action, receives a reward, and then moves to the next state. Designing the state, action, and reward determines what the agent learns. Today, we design it carefully.

Step 1: State & Action

Create a file named environment_under_rb.py One agent decision is one resource block. Choose which user to serve and with what power? 25, 50, or 100 percent? With 8 users and 3 power levels, there are 24 actions. The state contains 3 pieces of information for each user, SINR in this rb, the average rate already received, and the proportional fair metric, which is the log of the instantaneous rate divided by the average rate. Plus the position of the rb in the slot, totaling 25 numbers that are normalized to make it easier for the neural network to learn. The reset function creates a new network with randomly positioned users,

Step 2: Reward & Slot Transition

or a specific SID for fair evaluation. The step function translates the action into a user and power level, calculates the rate, and then gives the reward. The rate is divided by the average rate of that user, which is the proportional fair metric, minus beta multiplied by the power level as an energy cost. After 10 decisions, the slot is complete. Average rate is updated, channel changes due to fading, and after 200 slot episodes end.

Step 3: Random Policy vs PF

Create test_underscore environment file dot paye and run with two simple policies. The random policy only achieves total reward 1080, trupu 7.75 Mbps, and J0.634. The PF power full policy, which selects the largest PF metric from the state, achieves reward 1477, trupu 17.95 Mbps, and energy efficiency 1.795 Mbps. This ensures our environment is consistent with the simulator in episode 3.

Step 4: Reward & Action Selection

Let's look at the graphs. High PF reward at the beginning because all users start with the same average rate, then drops to equilibrium as the average rate forms, but remains above the random policy. The right diagram shows PF always selects the action with full power, indices 2, 5, 8, and so on. The agent's challenge in The QN is to find when small power is already sufficient.

Done!

Done. We have formulated resource block and power allocation as a mark of decision process in the gym environment, complete with state, action, and reward that balances fairness and energy. In the next episode, we will write and train a DeepQ Network agent. Working on a thesis, thesis, Or research in the field of telecommunications and artificial intelligence? Cerdas Inovasi Digital helps implement cellular network research, from the 5G simulator, channel model and SINR, proportional fair and waterfilling schedulers, until reinforcement learning with PyTorch, as well as opening the Python programming course for electrical engineering and telecommunications. Want to master the entire series, From the concept of resource block to evaluating the energy efficiency of the QN agent? All 6 episodes, complete with a step-by-step PDF guide and source code, Available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want just one class? Buy single class with lifetime access. Subscribe now on the premium page cerdasinovasidigital.com Contact our WhatsApp or visit cerdasinovasidigital.com Thank you for watching See you in the next episode

Ask the class material

Type a question or keyword; we find the part of this episode that explains it and jump the video there.

My notes

Notes are saved with the current video time and are only visible to you.

Questions & answers

Stuck on a step? Ask here; the instructor and other learners can answer.

No questions yet.

Your instructor

Cerdas Inovasi Digital Team

Software developers, BIM & 3D engineers, and AI practitioners

53 video classes1,291 episodes

We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.

See our clients →

Memuat

▶ Video Lesson

Recommended classes for you

Need more than a video?

Transcript

0:03Introduction

0:47Step 1: State & Action

3:07Step 2: Reward & Slot Transition

5:10Step 3: Random Policy vs PF

8:02Step 4: Reward & Action Selection

9:24Done!

Ask the class material

My notes

Questions & answers

Your instructor

▶ Video Lesson

Recommended classes for you

Need more than a video?

Transcript

0:03Introduction

0:47Step 1: State & Action

3:07Step 2: Reward & Slot Transition

5:10Step 3: Random Policy vs PF

8:02Step 4: Reward & Action Selection

9:24Done!

Ask the class material

My notes

Questions & answers

Your instructor

Introduction

Step 1: State & Action

Step 2: Reward & Slot Transition

Step 3: Random Policy vs PF

Step 4: Reward & Action Selection

Done!

Introduction

Step 1: State & Action

Step 2: Reward & Slot Transition

Step 3: Random Policy vs PF

Step 4: Reward & Action Selection

Done!