At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, nearly 150,000 network connections from the NSL-KDD intrusion dataset, grouped into normal and 4 categories of attacks. We also found a major challenge, 17 types of attacks only appear in the test data, such as the 0DI attack in the real world. Let's start. Hello, welcome to the research tutorial series, anomaly and intrusion detection in IoT networks with Hybrid Machine Learning. Billions of IoT devices such as cameras, sensors, and home routers become attractive targets for cyber attacks. Intrusion detection systems or IDS are responsible for identifying malicious traffic. In this 6-episode series, we build an IDS from scratch, intrusion dataset, preprocessing, feature selection, single model, hybrid model, until evaluation and dashboard monitoring.
Step 1: Installing the library
We open the terminal through the terminal menu, new terminal, then install pandas, scikit-learn, xgboost, torp, and matplotlib. The NSLKDD dataset is already available in the dataset folder in CSV format, specifically for premium customers.
Step 2: Four attack categories
Create a file named data_underscore_intrusion.py. NSLKDD contains 39 types of attacks grouped into 4 categories. The OS or Daniel of Service floods the target, for example Neptune and Smurf. Probe scans the network to find vulnerabilities, such as map and port swap. R2L, remote to local, tries to enter from outside, for example guessing passwords. U2R, user to root, tries to gain administrator privileges, such as buffer overflow. Dictionary to category reverses this list so that each type of attack can be directly mapped to its category. The load function reads CSV with pandas,
Step 3: Loading the dataset
adds a category column, normal or one of the 4 attack categories, as well as a binary attack column that is 1 for all types of attacks. The difficulty column from the dataset creator is discarded because it is not a traffic feature.
Step 4: Class composition & zero-day
Create a file explorasi.py and run it. The training data contains 125,973 connections and the test data 22,544 connections, each with 41 features. 53% of the training data is normal, 36% DOS, and 9% Probe. R2L is only 0.8% and U2R 0.04% in the training data, but in the test data R2L jumps to 12%. The most challenging part, 17 types of attacks only exist in the test data, covering 16.6% of the test connections. The model must detect attacks it has never seen before.
Step 5: Attack traffic patterns
Let's look at the graphs. Category composition on the lock scale shows extreme imbalance between normal and U2R. Distribution of source bytes shows many DOS have 0 bytes Because it only sends the initial connection packets. The feature map shows DOS has a high number of connections and a high level of SCN errors. These patterns will be learned by the model.
Done!
Done. We have now learned about 4 types of attacks, loading the NSL-KDD dataset, mapping 39 types of attacks, and identifying the class imbalance challenge as well as 0-Day attacks. In the next episode, we will process the traffic data. Currently working on a thesis, dissertation, or research in the field of network security and machine learning? Cerdas Inovasi Digital helps implement intrusion detection systems, from network traffic preprocessing, feature selection, random forest and XGBoost, up to autoencoder for 0-Day attacks and monitoring dashboard, as well as opening Python programming courses for electrical engineering and computer science. Want to master the entire series? From intrusion datasets to hybrid systems ready for monitoring? All 6 episodes, complete with step-by-step PDF guides dataset, and source code, are available for premium tutorial subscribers. Subscribe starting at Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want just one class? Buy single classes with lifetime access. Subscribe now on the premium page at cerdasinovasidigital.com. Contact our WhatsApp, or visit cerdasinovasidigital.com. Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
53 video classes1,290 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
