At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
This is the final result, we processed employee data with pandas, calculated the average salary per division, and then displayed it as four graphs at once with Matplotlib. These three libraries, Nampai, Pandas, and Matplotlib, are essential tools before entering machine learning. Let's start. Hello, welcome back to the Machine Learning with Python series, episode two. Before training a model, we must be proficient in data processing. In this episode, we install the required libraries, learn Nampai for fast numerical computation, Pandas for processing table data like Excel, and Matplotlib for creating graphs. All examples are directly typed and run in VS Code.
Step 1: Installing libraries
First step, open the terminal via the terminal menu, new terminal, and install all libraries with one command, PIP install Nampai Pandas Matplotlib scikit-learn. Since these are already installed on this computer, PIP only informs that the requirements are already met. We check the version with the command python-c, then create a file Nampai underscore dasar dot py. Nampai provides arrays, which are a collection of numbers much faster than Python lists.
Step 2: NumPy Arrays and Vector Operations
We create one-dimensional and two-dimensional arrays. The shape attribute tells us the size, for example, two rows and three columns. What's special is that operations on arrays apply to every element without the need for loops. X plus Y adds corresponding elements, X times 10 scales everything, and dot calculates the dot product, an operation that will be used in every linear model.
Step 3: Indexing and Statistics in NumPy
Next, indexing and statistics. We create a matrix of values for three students and three subjects. With a boolean mask, values above 80 can be extracted with just one line. The min function can be calculated per column with the same action as zero, or per row with the same action as one. Argmax gives the index of the student with the highest average. We also create random numbers with SID, so the results are always the same every time it is run.
Step 4: pandas DataFrame and CSV Files
Now pandas. Create a file pandas underscore dasar dot py. We create a data frame containing data for six employees. Name, city, age, salary, and division. There's one missing age, intentionally, like data in the real world. We save it to a CSV file then read it again with read CSV, the most common format used in machine learning. Then we look at the first five rows, summary statistics with describe, and the number of missing data.
Step 5: Filtering, New Columns, and groupby
Pandas is very powerful for filtering and summarizing data. We take employees with a salary above 8 million, then IT division employees living in Jakarta with two conditions at once. We fill the missing age with the median, and we add a new column for annual salary. Finally, group by, similar to pivot table in Excel, to calculate the average salary per division.
Step 6: Four Plots with matplotlib
Finally, visualization. Create a file graphic dot py. With PLT subplots we make four graphs in one window. Line graphs for sine and cosine functions, scatter plot for random points, histogram for height forming a bell curve, and bar chart for average salary per division from the CSV data above. The function tick layout tidies up the spacing between graphs.
Done!
Done. We have installed the libraries, used arrays and vector operations with numpy, processed tables with pandas, and created various graphs with matplotlib. In the next episode, we learn the mathematics behind machine learning, and train a regression model from scratch with gradient descent without any machine learning library at all. Want to master machine learning and data science? Cerdas Inovasi Digital offers Python, machine learning, and AI courses from scratch, prediction model development services, data analysis, and AI applications for business or research, as well as selling complete source code of this tutorial. Want to master the entire series, from introduction to machine learning up to deploying the model to the web? All 18 episodes, complete with PDF step-by-step guides, are available for premium tutorial subscribers. Subscribe starting from Rp69.000 per month, save 28% with the 12-month package, or pay once Rp999.000 for lifetime access. Want a single class only? Buy a single class with lifetime access. Subscribe now at the premium page cerdasinovasidigital.com. Contact us via WhatsApp, or visit cerdasinovasidigital.com. Thank you for watching, see you in the next episode.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
52 video classes1,276 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
