Analytics & ML · Study deck

Feature Scaling Lab: Preprocessing Workflow

Start with one saved copy of the raw data.

Data Dora is your guide for this deck.

dataqualitynormalization

Data Dora, the module guide, in a scene from this chapter.

After studying this chapter

Learning objectives

You will be able to:

Apply Normalization Techniques: Implement min-max scaling, Z-score normalization, and robust scaling for multi-sensor data fusion
Compare Normalization Methods: Evaluate which scaling approach suits each downstream use case (neural networks, clustering, visualization)
Implement a Complete Pipeline: Build and test an end-to-end data quality system on an ESP32 microcontroller
Assess Data Quality Metrics: Calculate and interpret validation rates, outlier counts, and imputation statistics

Major section

Start With the Story

The values use different units and ranges.

If the team scales all records before it separates training data from test data, the test set quietly shapes the model.
Scaling cannot repair a bad sensor or prove that an alert is useful.
It only changes how values enter the next step.

Major section

In 60 Seconds

You will compare Min-Max normalisation, Z-score standardisation, and robust scaling on data with outliers, and see the impact on a simple classifier.

The mathematical gist.: The simulator's 10000 × (4095/adc: 1) is the voltage-divider equation solved backward.

Major section

For Beginners: Why Normalize Data?

If you feed both to a machine learning model without normalizing, the model will think light is 2000x more important simply because the numbers are bigger!

Key question this chapter answers: "How do I prepare multi-sensor data so it can be combined and analyzed fairly?".

Major section

Lab Overview

In this hands-on lab, you will implement a complete data quality preprocessing pipeline on an ESP32 microcontroller.

The simulation demonstrates real-world techniques for handling sensor data problems including outliers, missing values, noise, and the need for normalization.
Sensor Data Validation: Implementing range checks and rate-of-change validation.

Try it: Lab Overview in the chapter

Deck summary

Key takeaways

The values use different units and ranges.

You will compare Min-Max normalisation, Z-score standardisation, and robust scaling on data with outliers, and see the impact on a simple classifier.
If you feed both to a machine learning model without normalizing, the model will think light is 2000x more important simply because the numbers are bigger!
In this hands-on lab, you will implement a complete data quality preprocessing pipeline on an ESP32 microcontroller.

Retrieval practice

Recall check

Data Dora says: answer from memory, then check your reasoning.

Q1A humidity dataset [45,48,50,52,55,99,47,51,49,50] contains a stuck-sensor outlier of 99. Which scaling method is least distorted by it?

ARobust scaling, which centers on the median and scales by the IQR rather than the mean

BMin-max scaling, because it stretches the data to a fixed 0-to-1 range

CZ-score standardization, since subtracting the mean removes any outlier effect

DNo scaling at all, because raw humidity values are already comparable

Show answer

Answer: A Robust scaling uses median and IQR, so a single stuck-sensor outlier barely affects the transformed values.

Print reference

Answers

Answer key.

A · Robust scaling uses median and IQR, so a single stuck-sensor outlier barely affects the transformed values.