← Back to the chapter · Next detail →
Technical detail · IoT Machine Learning Fundamentals
Model boundaries, dimensions and overfitting
Model families draw different boundaries through that feature space. A nearest-neighbour classifier assigns a new example by looking at nearby labeled examples, so the choice of k and the feature scale can change the answer. Decision trees split on one feature test at a time; random forests combine many such trees so one brittle split does not dominate. Support vector machines look for a separating margin, can extend multi-class work through one-versus-rest classifiers, and can use kernels to make nonlinear separation possible after mapping examples into a richer feature space. These names are useful only when the release record also preserves the feature set, scaling rule, validation split, latency budget, and fallback behavior.
More dimensions can make classes easier to separate, but they can also create false confidence. A model is strongest in the region represented by its training examples; outside that region there are no guarantees unless validation deliberately held out the relevant user, device, site, season, or operating mode. High-dimensional feature sets therefore need the same discipline as simple ones: test the deployment boundary, watch for drift, and route unsupported inputs to review instead of forcing every point into the nearest known class.
Overfitting is the warning sign that the learned boundary has memorized special cases instead of learning a decision that travels. A high-degree curve can pass close to every training point while swinging away from held-out examples; a high-dimensional classifier can draw tiny pockets around examples that look convincing in training but fail in deployment. Prefer the simplest boundary that preserves the required signal, then prove it with held-out users, devices, sites, seasons, or time periods that match the intended generalization claim.