Normalisation and Standardisation of Process Sensor Data Before Manufacturing Product Quality Modelling

Project info

• Field of studyIndustrial Engineering and Production
• LevelUndergraduate thesis (S1)
• Method typeData Quality
• Estimated build time± 3 working days
• Toolsscikit-learn MinMaxScaler, StandardScaler, RobustScaler
• PriceUS$19 - US$119

Research goal

Check and improve the data quality of product quality score together with process temperature and process pressure using Data Normalisation and Standardisation.

Variables

Example result

• 0.57kNN accuracy (raw)
• 0.661kNN accuracy (z-score)
• 0.644kNN accuracy (min-max)
• 1823.635SD ratio (raw)
• 0Mean after z-score
• 1SD after z-score

Variable scales differ by up to 1823.6x; after standardisation, kNN accuracy moves from 0.570 to 0.661 (min-max 0.644).

Computed from illustrative synthetic data, not from a real study.

About Data Normalisation and Standardisation

Normalisation (min-max) and standardisation (z-score) put variables on the same scale so none dominates because of its units. They are required before distance- or penalty-based methods such as k-NN, SVM, K-Means, and regularised regression.

Steps

1. 1Check the range and units of each variable
2. 2Split into training and test sets first
3. 3Fit the scaler (MinMaxScaler, StandardScaler, RobustScaler) on training data
4. 4Apply the same scaler to test data
5. 5Re-check the mean, standard deviation, or range of the result
6. 6Compare distributions before and after

Assumptions to check

• The scaler is learned from training data only to avoid leakage
• RobustScaler is more suitable when outliers exist
• Scale changes but distribution shape does not

Reading and reporting the results

Explain the chosen scaling method and why, then show statistics before and after. State that scaling parameters are stored so new predictions are treated the same way.

Example Python code

A short starting script with this project's variable names. The delivered version is adapted to your data and fully commented.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import MinMaxScaler, StandardScaler, RobustScaler
from sklearn.model_selection import train_test_split

rng = np.random.default_rng(38)
n = 300
process_temperature = rng.normal(1000, 200, n)       # large-scale variable
process_pressure = rng.exponential(2.0, n)        # small-scale, skewed variable
process_pressure[:3] += 30                        # outliers
product_quality_score = np.column_stack([process_temperature, process_pressure])

X_tr, X_te = train_test_split(product_quality_score, test_size=0.3, random_state=38)
scalers = {"MinMax": MinMaxScaler(), "Standard": StandardScaler(), "Robust": RobustScaler()}

print("Raw train mean:", X_tr.mean(axis=0).round(2), "std:", X_tr.std(axis=0).round(2))
fig, ax = plt.subplots(1, 4, figsize=(14, 3.5))
ax[0].boxplot(X_tr); ax[0].set_xticklabels(["x1", "x2"]); ax[0].set_title("Raw")
for a, (name, sc) in zip(ax[1:], scalers.items()):
    tr = sc.fit_transform(X_tr)
    te = sc.transform(X_te)
    print(name, "train mean:", tr.mean(axis=0).round(2), "std:", tr.std(axis=0).round(2),
          "test min:", te.min(axis=0).round(2))
    a.boxplot(tr); a.set_xticklabels(["x1", "x2"]); a.set_title(name)
plt.tight_layout()
plt.show()

Packages and prices

Choose how much help you need. Each higher package includes everything in the lower ones. Prices exclude 11% VAT.

• Payment: 50% down payment before work starts, balance after the deliverables are handed over
• Tax: Prices exclude 11% VAT
• Data and academic integrity: Results are computed from your own research data; you write and stand behind your manuscript, we help with the analysis and your understanding of it
• Revisions: Up to 2 revisions are included in the price (Package 3: 3 rounds)

Frequently asked questions