All classes ANFIS Metaheuristics Time Series Forecasting Syllabus · Episode 6 of 6

Metaheuristic ANFIS Time Series Forecasting Class #6: RMSE/MAPE Evaluation, Diebold-Mariano Test, and Forecast Visualization

4 views

Tutorial on evaluating forecasts with RMSE, MAE, sMAPE, and Diebold-Mariano test from scratch using Python. Episode 6, the closing part of the ANFIS metaheuristics time series forecasting series.

Software: Python (NumPy, statsmodels, PyTorch, Matplotlib)
Level: Advanced
Duration: ± 8 min
Series total: 59 min · 6 videos
Published: October 25, 2026

Episode 6 of 6 · view syllabus

Subscribe ▶ Watch the video lesson1 plays

Checking access...

Want to go deeper?

Join one of our courses or ask us directly on WhatsApp.

View the full 6-episode syllabus View ANFIS Time Series Courses Ask on WhatsApp

Recommended classes for you

Picked from the software, topic, and level of “Metaheuristic ANFIS Time Series Forecasting Class #6: RMSE/MAPE Evaluation, Diebold-Mariano Test, and Forecast Visualization”.

6 episodesFree preview

Python · Intermediate–Advanced

Electric Load Forecasting with LSTM

6 video lessons with PDF guides, 53 min in total. Starts with: Hourly Electricity Load Data: Daily, Weekly, and Seasonal Patterns.

53 min 0 0

View class

18 episodesFree preview

Python · Beginner–Advanced

Machine Learning Python

18 video lessons with PDF guides, 3 h 41 min in total. Starts with: Introduction to Machine Learning & First Model.

3 h 41 min 61 0

View class

NSGA-II MOPSO Hybrid Multi-Objective Optimization

6 episodesFree preview

Python · Intermediate–Advanced

NSGA-II MOPSO Hybrid Multi-Objective Optimization

6 video lessons with PDF guides, 59 min in total. Starts with: Multi-Objective Optimization: Pareto Front, Dominance, and Test Functions ZDT/DTLZ.

59 min 9 0

View class

100 episodesFree preview

Claude · Beginner–Advanced

Claude from Zero to Expert

100 video lessons with PDF guides, 15 h 1 min in total. Starts with: What Claude Is and How It Differs from ChatGPT and Gemini.

15 h 1 min 62 0

View class

50 episodesFree preview

Agentic AI · Beginner–Intermediate

Agentic AI Without Coding

50 video lessons with PDF guides, 14 h 29 min in total. Starts with: Chatbots, Automation, and AI Agents: What's the Difference?.

14 h 29 min 66 0

View class

6 episodesFree preview

Python · Intermediate–Advanced

Brain Tumor Segmentation with U-Net

6 video lessons with PDF guides, 1 h 10 min in total. Starts with: Brain MRI Dataset & Tumor Mask: Loading, Visualization & Patient-wise Splitting.

1 h 10 min 28 0

View class

Deep Learning CNN-LSTM Pneumonia Detection

6 episodesFree preview

Python · Intermediate–Advanced

Deep Learning CNN-LSTM Pneumonia Detection

6 video lessons with PDF guides, 57 min in total. Starts with: Chest X-ray Dataset: Loading, Exploration & Data Splitting.

57 min 22 0

View class

6 episodesFree preview

Python · Intermediate–Advanced

Transfer Learning for Leaf Disease

6 video lessons with PDF guides, 1 h 5 min in total. Starts with: PlantVillage Dataset: Folder Structure, Class Distribution & Data Split.

1 h 5 min 9 0

View class

Need more than a video?

Other ways we can help with Python.

Learn with a mentor

Scheduled AI & Machine Learning courses with live guidance and a certificate.

Course

Multi-Objective Optimization Course: NSGA-II, MOPSO & Hybrid Algorithms (Python)

Course

ECG Arrhythmia Classification Course with Wavelet & Deep Learning (PyTorch)

Course

ANFIS + PSO/GA Course for Non-Linear Time Series Forecasting (Python)

See all in AI & Machine Learning →

Have it built for you

No time to learn Python? Our team can do the project for you.

Service

Python Machine Learning — Prediction Models, Customer Segmentation, Fraud Detection & AI Apps

Service

AI Image Creation with Stable Diffusion — Unlimited Custom Visual Production

★ 4.9

Service

AI Image & Video Creation with Leonardo AI — All-in-One Visuals for Business

★ 4.9

See all in AI & Machine Learning →

Get notified about new Python classes

At most one email a week. Unfollow anytime with one click.

Transcript

The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.

Show transcript

Introduction

Here is the final result, a complete evaluation with the Diebold-Mariano statistical test. On Mackey Glass, Anvis has a significant advantage with P10-13. On Sunspot, ARIMA is slightly more superior significantly, while Anvis and LSTM are not significantly different. Let's start. Hello, welcome to the final episode of the Anvis Meta Heuristik series for time series forecasting. A slightly smaller RMSI does not necessarily mean a better model; it could just be a coincidence on a particular test data. The Diebold-Mariano test answers this question statistically, and it has become the standard in forecasting journals. Today we write it from scratch.

Step 1: Diebold-Mariano Test

Create a file named evaluation.py The Diebold-Mariano function calculates the squared error difference between the two models at each point, then tests whether the average is different from 0. The variance accounts for autocorrelation up to H-1 lag for the forecast steps, and the Harvey, LeBourne, and Newball corrections improve the test size for small samples. A negative statistic means the first model is more accurate.

Step 2: Metrics & Significance Test

The main part loads all forecasts, returns them to the original format, then calculates RMSI, MAI, and SMAPE. We use SMAPE, not the regular MAPE, because the Sunspot data can have values almost 0, causing MAPE to explode. Then Anvis is tested against each benchmark with Diebold-Mariano. Run it. On Mackey Glass, Anvis PSO has an RMSI of 0.0026 with SMAPE only 0.22%, compared to LSTM 2.79% and ARIMA 3.98%. The DM statistic against ARIMA is -7.62 and against LSTM is -10.6, both are highly significant. On Sunspot in solar spot units, ARIMA RMSI is 17.4, Anvis 20.1, and LSTM 20.6. The DM Anvis against ARIMA is 2.05 with P=0.043, meaning ARIMA is significantly more accurate, while Anvis and LSTM are not significantly different with P=0.72. Continue with the graphs, RMSI, box plot of errors, and diagram of actual versus forecast.

Step 3: Error & Forecast Visualization

Let's see the graphs. The error box plot for Mackey Glass Anvis is very narrow around 0, and the points in the actual versus forecast diagram are aligned along the diagonal line. In the sunspot, the error distribution of the three models is similar, with long tails at the peaks of the cycles. The researcher's conclusion is that the metaheuristic Anvis is highly superior for non-linear time series with rich data, while for short time series, the linear model remains a viable option. Done, and the series is complete.

Done!

We have built a complete forecasting research, time series analysis, Sugeno fuzzy, Anvis from scratch, PSO and GE optimization, comparison with LSTM and ARIMA, up to evaluation and Diebold-Mariano test. For further research, try Anvis with rule reduction such as subtractive clustering, hybrid Anvis with wavelength decomposition, multi-step forecasting, or real data such as electrical load and rainfall. Thank you for following this series. Currently working on a thesis, dissertation, or research in the field of forecasting and artificial intelligence? Cerdas Inovasi Digital helps implement forecasting models, from Sugeno fuzzy and Anvis systems, parameter optimization with PSO and genetic algorithms, up to comparison with LSTM and ARIMA as well as Diebold-Mariano tests, and also opens Python programming courses for engineering. Want to master the entire series, from chaotic time series up to statistical evaluation? All 6 episodes, complete with step-by-step PDF guides, datasets, and source code, are available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want just one class? Buy a single class with lifetime access. Subscribe now at the premium page cerdasinovasidigital.com Contact our WhatsApp or visit cerdasinovasidigital.com Thank you for watching, see you in the next episode. Thank you for watching,

Ask the class material

Type a question or keyword; we find the part of this episode that explains it and jump the video there.

My notes

Notes are saved with the current video time and are only visible to you.

Questions & answers

Stuck on a step? Ask here; the instructor and other learners can answer.

No questions yet.

Your instructor

Cerdas Inovasi Digital Team

Software developers, BIM & 3D engineers, and AI practitioners

53 video classes1,296 episodes

We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.

See our clients →

Memuat

Recommended classes for you

Need more than a video?

Transcript

0:03Introduction

0:49Step 1: Diebold-Mariano Test

2:22Step 2: Metrics & Significance Test

4:46Step 3: Error & Forecast Visualization

6:41Done!

Ask the class material

My notes

Questions & answers

Your instructor

Recommended classes for you

Need more than a video?

Transcript

0:03Introduction

0:49Step 1: Diebold-Mariano Test

2:22Step 2: Metrics & Significance Test

4:46Step 3: Error & Forecast Visualization

6:41Done!

Ask the class material

My notes

Questions & answers

Your instructor

Introduction

Step 1: Diebold-Mariano Test

Step 2: Metrics & Significance Test

Step 3: Error & Forecast Visualization

Done!

Introduction

Step 1: Diebold-Mariano Test

Step 2: Metrics & Significance Test

Step 3: Error & Forecast Visualization

Done!