17 Hardware Optimization Strategies

17.1 Start With the Block That Burns Energy

Picture a battery sensor that wakes, reads a value, sends it, and sleeps. One part may use most of the energy. It may be the radio, a memory copy, a voltage converter, or a block that never turns off.

Find that costly part first. Measure one full work cycle, including sleep and wake time. Change one part only when the record shows that it controls the budget. A faster chip is not an answer by itself.

Now check the price of the change. A special block can finish work quickly but waste power while idle. A wider data path can save time but add cost and design risk. A new power part can lose its gain at light load.

The simple cycle cannot pick a processor, memory path, power rail, or custom chip. It also cannot prove heat, update, or supply risk. Those choices need real loads and a before-and-after test on the whole device.

Use the Practitioner sections to select blocks and review the work path. Use Under the Hood for current, rail loss, memory traffic, and special hardware. The deeper checks explain when the simple rule holds and when a trade-off changes the result.

Run a plain before-and-after check. Charge the same battery. Use the same work. Use the same room. Mark sleep current. Mark wake time. Mark active current. Count each data move. Count each radio send. Measure heat. Repeat the run. Change one block. Run it again. Compare total charge. Compare missed work. Compare start time. Compare board cost. Keep the old design as a base. Reject a gain that harms safety. Reject a gain that harms repair. Accept only the whole-device result.

A hardware optimization is useful only when it reduces the part of the workload that dominates the budget. An accelerator, buck regulator, DMA path, or peripheral choice can save energy, but it can also add idle loss and integration cost.

Begin with the measured bottleneck, then choose hardware that reduces active time, transfer cost, or wasted conversion in the dominant regime.

Phoebe’s Field Notes: The Physics Hiding Inside “$\mu$A/MHz”

The mathematical gist. A core specified at 50 microamps per MHz on a 1.8 V rail behaves like about 27.8 pF of switched capacitance. At 48 MHz it draws 2.40 mA and 4.32 mW. Halving frequency alone gives 2.16 mW, but halving frequency while lowering the rail to 1.2 V gives 0.960 mW, a 77.8% saving because dynamic power follows $CV^2f$.

Math Bridge · guided foundationsWhy does lowering voltage save more than slowing the clock?Let Battery Bruno recover switched capacitance from the datasheet and compare frequency-only scaling with DVFS. Learn the maths

17.2 Hardware Optimization Strategies

Hardware optimization is not only “choose a faster chip.” In IoT systems, it usually means matching each workload to the cheapest safe hardware path that meets timing, energy, memory, and reliability constraints.

Good hardware optimization often happens before a board is finalized:

• choose a processor family with enough sleep modes, wake sources, and peripheral support
• offload repeated transfers to timers, DMA, or communication engines
• reduce memory movement with better buffering and data layout
• select an accelerator only when the algorithm is stable and the measurement justifies it
• define power domains so unused parts can be switched off

17.3 Learning Objectives

By the end of this chapter, you will be able to:

• Compare general-purpose processors, hardware peripherals, DSP-style blocks, reconfigurable logic, and custom silicon.
• Explain why data movement and memory bandwidth can dominate hardware optimization.
• Use selection gates for acceleration, power domains, clocks, and platform changes.
• Identify when a hardware change is premature or too risky.
• Build validation evidence for before/after hardware optimization decisions.

17.4 Hardware Choices As A Spectrum

Hardware options form a spectrum from flexible software control to highly specialized circuits.

Read Figure 17.1 from CPU to ASIC as a trade, not a ranking. Moving right can turn a stable kernel into dedicated parallel hardware, but each step gives up some ability to repair changing requirements in software.

The figure in Figure 17.1 puts the CPU at the end with the lowest non-recurring engineering cost and the most software flexibility, but it provides the baseline performance and lowest efficiency for regular arithmetic. A DSP narrows the design around multiply-accumulate work while remaining programmable. An FPGA trades more engineering effort and unit cost for gate-level parallelism and field reconfiguration. An ASIC reaches the highest performance and volume efficiency only after accepting fixed function and potentially very high design cost. The decision is therefore workload- and volume-dependent: specialize only as far as measured performance or energy evidence justifies.

The “general processor” end of the spectrum is not one part. Review boards often compare a small handful of general-purpose MCUs before picking one, and the spread in flash, RAM, and unit cost across otherwise similar Cortex-M parts is large enough to change the design:

That is roughly a 5.5x cost spread inside one review, before a single-board computer is even considered. The gap to the next rung up the spectrum is larger still: a Raspberry Pi-class single-board computer typically draws on the order of 700-1200 mA at 5 V (3.5-6 W) even before peripherals, while a duty-cycled microcontroller can idle at a few milliwatts and hold active current in the tens of milliwatts. Choosing the smallest part that still meets flash, RAM, and peripheral needs is a real energy and cost decision, not a rounding error — and it is the evidence a “general processor” review row in the ledger above should record, not just a feature checklist.

17.5 Selection Gates

Use gates before moving from software or peripheral offload to a specialized hardware path.

The diagram in Figure 17.2 turns an accelerator proposal into a six-gate review. Start by fixing the workload and measuring the actual limit; otherwise the team may optimize an unstable algorithm or the wrong bottleneck.

In Figure 17.2, the second gate separates compute time from transfer and memory cost. The third then asks whether the proposed data path avoids copies and wakeups; a fast kernel behind an expensive transfer can lose at system level. Only after those measurements pass should the review accept field-update risk and decide whether product volume justifies specialization. The sixth gate closes the loop with before-and-after timing, energy, and correctness evidence on the real workload.

17.6 Data Movement And Memory

Many hardware optimization failures happen because the compute block is faster, but data cannot reach it efficiently. A fast accelerator that requires repeated copies, cache flushes, wakeups, or polling can lose the energy benefit.

Trace one sensor or radio burst through Figure 17.3. The useful question is not merely how quickly the accelerator runs, but how much of the route can complete while the CPU remains asleep.

In Figure 17.3, the peripheral buffer absorbs arrival jitter so a burst does not force an immediate CPU wake. DMA then moves a block into shared memory without a software copy. From that bank, an accelerator can perform the regular kernel work and leave the CPU to review only the result or exception. Each extra copy, interrupt, cache flush, or polling loop breaks part of this sleep opportunity, so measure wake count and bytes moved alongside accelerator execution time.

17.11 Power Domains, Clocks, And Peripherals

Before applying the specification, inspect the real shunt resistor (current-sense) below: its package, terminals, scale, and installation context are part of the engineering evidence.

Carry those visible constraints into the surrounding analysis; the abstract symbol or capability name does not capture mounting, wiring, protection, or service access.

Hardware optimization also includes turning hardware off correctly.

17.12 Worked Review: Streaming Sensor Node

Suppose a sensor node collects short bursts of high-rate samples and then sends a compact event summary. A software-only prototype works, but the energy trace shows the CPU stays awake while it copies samples and waits for transfers.

17.13 Review Checklist

Before accepting a hardware optimization:

• The workload and target requirement are stable enough for the chosen hardware path.
• The baseline includes timing, current, memory movement, wake behavior, and error cases.
• The proposed hardware targets a measured bottleneck.
• Data movement into and out of the hardware path is included in the measurement.
• Power-domain and clock-state transitions are tested.
• Correctness is checked with reference vectors, boundary cases, and long-run stress.
• The update and rollback story is clear.
• The final evidence uses the real duty cycle, not only a kernel benchmark.

17.14 Knowledge Check

17.15 Match Hardware Item To Purpose

17.16 Order The Hardware Optimization Process

17.17 Label The Hardware Optimization Record

17.18 How You Make The Rail Decides How Much You Waste

Every battery device converts the cell voltage into the rails its chips need, and the converter you choose can quietly waste a large fraction of the energy. A low-dropout linear regulator (LDO) is simple and quiet, but its efficiency is roughly the ratio of output to input voltage: eta = Vout / Vin. It passes the full load current through from the battery and burns the leftover voltage as heat. A well-chosen switching buck can reach roughly 85-95% near its intended operating point, but that figure is not constant: conversion ratio, load current, switching mode, quiescent current, and conduction and switching losses all move the efficiency.

The regulator topology becomes a physical heat and headroom choice at the component. The photograph in Figure 17.4 provides a concrete linear-regulator example before the chapter compares what happens to voltage that does not reach the load.

Read Figure 17.4 from the 7809 package marking to the three leads, then to the metal tab and its mounting hole. “7809” identifies a fixed 9 V member of the linear 78xx family; the leads carry input, ground, and output, while the tab provides a thermal path rather than an energy-conversion stage. This is why the rail discussion must calculate dropped-voltage times load-current and verify temperature instead of treating the part’s simple pinout as evidence of efficiency.

The map in Figure 17.5 follows energy from an imperfect source to several very different loads. Use it to locate where voltage headroom becomes protection loss, regulator loss, ripple, or useful load power.

In Figure 17.5, the source column spans a 3.7 V LiPo, 5 V USB, higher-voltage solar input, and a 12 V adapter, so no single regulator topology fits every case. Protection first makes that raw input survivable; regulation then selects LDO, buck, boost, or buck-boost according to the required rail and source range. Filtering controls the ripple delivered to the circuit, but its capacitors and ferrites do not recover energy already lost in regulation. Finally, the load column shows why the rail must be sized from concurrent peaks: a 500 mA motor and 200 mA radio can dominate an 80 mA MCU and 5 mA sensor even when their average duty cycles are low.

The consequence is that an LDO draws the same current from the battery as the load draws, no matter how much voltage it is dropping, while a buck draws less battery current than the load whenever the input voltage is above the output. The wider the gap between battery voltage and rail voltage, the more a linear regulator wastes and the more a switcher saves.

Scale the loss over time before dismissing it. If the 100 mA rail is active for 5 minutes each hour, it runs 2 hours per day. The LDO waste is 90 mW x 2 h = 180 mWh per day. The 90% buck loses 37 mW x 2 h = 74 mWh per day. The difference, 106 mWh, is about 29 mAh from a 3.7 V cell each day, large enough to change the battery-size decision.

17.19 The Regulator Choices

17.20 Overview Knowledge Check

17.21 An LDO Passes The Load Current; A Buck Reduces It

Compare regulators by the battery current they draw, not just efficiency. For an LDO, I_battery = I_load. For a buck, I_battery = P_load / (Vin x eta), which falls below the load current when Vin exceeds Vout.

17.22 Worked Example: 100 mA At 3.3 V From A Full Li-ion (4.2 V)

The load delivers 330 mW at 3.3 V. Compare an LDO with a 90% buck.

• LDO: battery current = 100 mA; battery power = 4.2 V x 100 mA = 420 mW; efficiency 3.3/4.2 = 79%; 90 mW wasted as heat.
• Buck at 90%: battery power = 330 mW / 0.90 = 367 mW; battery current = 367 mW / 4.2 V = 87 mA; 37 mW lost.
• Result: the buck draws 87 mA versus the LDO's 100 mA, about 13% less battery current during this high-current phase, and the advantage grows as the battery sits higher above 3.3 V.

From a nearly empty 3.7 V cell the gap narrows: the LDO reaches 3.3/3.7 = 89%, almost matching the buck. So the buck's win is largest exactly when the battery is full and the voltage gap is widest.

That comparison also explains why "90% efficient" is not enough by itself. At 4.2 V, the buck input current is 367 mW / 4.2 V = 87 mA. At 3.7 V, the same 367 mW input power requires 99 mA, almost the LDO's 100 mA. A battery-life estimate should therefore integrate the regulator choice across the discharge curve, or at least test full, nominal, and low-battery cases instead of using one current number.

17.23 Regulator Comparison Ledger

17.24 Practitioner Knowledge Check

17.25 Efficiency Is Load-Dependent, So Pick For The Dominant Regime

Regulator efficiency is not one number; it changes with load. A buck is excellent at the high currents of a radio burst, but at the microamp currents of sleep its own controller quiescent current can dwarf the load. Suppose a buck controller draws 15 uA of quiescent current and the sleep load is 10 uA. The regulator now consumes 25 uA to deliver 10 uA - an effective efficiency near 40%. A low-quiescent LDO drawing 1 uA delivers the same 10 uA at 11 uA total, about 91% effective. At light load the simple LDO wins decisively.

Over one day, that sleep choice alone is visible. A 25 uA buck-fed sleep rail for 23.76 hours consumes about 594 uAh. An 11 uA low-Iq LDO sleep rail consumes about 261 uAh over the same interval, saving roughly 333 uAh per day. If the product reports rarely, that sleep-rail saving can be larger than the active burst optimization; if it reports constantly, the high-current buck path may dominate instead.

This matters because a duty-cycled device spends almost all of its time asleep, so the sleep regulator, not the active one, often dominates the energy budget. A common mistake is to select a single high-efficiency buck for the radio peak and then pay its quiescent current through months of sleep. The better designs pick the regulator for the regime that dominates the energy: a buck for the high-current active phase, and a low-quiescent LDO or a converter with a light-load pulse-frequency mode for the long sleep. The same finish-and-sleep logic explains hardware accelerators: a crypto or DSP block that completes an operation in far fewer cycles than the CPU lets the whole system return to that low-power sleep sooner, so the accelerator is an energy win even when its instantaneous power is higher.

17.26 Choosing By Regime

17.27 Under-the-Hood Knowledge Check

17.28 ASIC Specialization Axes

An ASIC earns its efficiency by deleting generality. That deletion must follow evidence, because a circuit frozen around the wrong workload cannot be repaired with a compiler flag. Treat specialization as a chain from measured software to silicon, not as a menu of clever instructions.

The development chain is:

1. Profile representative source. Capture hot kernels, data widths, memory traffic, branch behavior, deadlines, and worst-case inputs.
2. Explore architectures. Compare a tuned MCU, peripheral offload, DSP, reconfigurable logic, and custom logic using the same workload and accuracy contract.
3. Define the programmer contract. Select the instruction set, registers, data types, memory model, exceptions, assembly syntax, and compiler intrinsics that firmware will actually use.
4. Design the microarchitecture. Map that contract onto datapaths, functional units, memories, interconnect, and control.
5. Build the toolchain and firmware together. An instruction the compiler cannot emit safely is not a usable accelerator feature.
6. Benchmark the full path. Include instruction fetch, memory movement, stalls, conversion boundaries, and idle power—not only the arithmetic block.

Five coordinated axes shape the result:

Instruction fusion shows how the axes interact. Replacing load, multiply, add, store with a fused kernel only saves energy if operands arrive without extra copies, the accumulator width preserves accuracy, the register file can supply the ports, and the compiler recognizes the pattern. Otherwise, the “special” instruction waits on memory or forces marshaling that consumes the saving.

Memory banking has the same dependency. Four single-ported banks can serve four simultaneous accesses only when addresses distribute across different banks. If every lane addresses the same bank, the requests serialize. Profiling must therefore record access strides and conflicts, not just total bytes. Cache choices are equally application-specific: a streaming kernel may benefit more from scratchpad DMA than from a large associative cache whose tags and replacement logic switch on every access.

The exit criterion is a four-column ledger—performance, energy, area, and toolchain consequence—for every proposed feature. Specialize only when the evidence shows a stable benefit across representative and worst-case workloads.

17.29 Heterogeneous Run-State Migration

Heterogeneous multicore systems can implement the same instruction architecture on “little” efficient cores and “big” fast cores. Compatibility lets a thread migrate, but migration is not free: caches may be cold, state must be transferred, and the receiving core may need a voltage or frequency transition.

For a candidate migration, compare completion energy rather than instantaneous power:

Move only when $E_{move}<E_{stay}$ or when the little core would miss the deadline. A big core can consume more power yet less energy if its shorter runtime is enough to overcome transition and refill costs. Conversely, bouncing a short task between cores can cost more than simply finishing it where it started. Practical schedulers therefore use utilization windows or hysteresis instead of migrating on every brief load spike.

17.30 Always-On Vision Accelerator Data Path

Always-on vision exposes both sides of specialization: arithmetic per pixel and bytes moved per frame. Historical 4 mm-class object-detection silicon demonstrated histogram-of-oriented-gradients (HOG) and deformable-part-model (DPM) processing below roughly 1 nJ per pixel—comparable in scale to dedicated video-codec work in that implementation. Treat those numbers as a measured design point, not a promise for a different resolution, process, model, or memory system.

The data path explains the saving:

1. A conventional imager digitizes the full pixel array.
2. Raw or lightly processed pixels cross an interface into processor memory.
3. A feature extractor repeatedly reads neighborhoods to compute gradients, magnitudes, orientation bins, and normalized HOG cells.
4. A detector consumes the feature tensor and reports candidate objects.

Sensor-adjacent extraction moves step 3 beside the imager. Row buffers retain the small neighborhood needed for gradients, a compact accelerator emits feature cells, and only that feature stream crosses to detector memory. In the cited implementation, this reduced the transferred representation by about 20 times. The important result is not the portrait of a detection chip; it is the boundary change from pixel-space transmission to feature-space transmission.

For a frame of $N_p$ pixels at $b_p$ bits per pixel, raw transfer volume is

If feature extraction reduces the representation by factor $r$, then $D_{feature}=D_{raw}/r$. The transfer-energy saving is

With $r=20$, 95% of the raw transfer bits disappear. The accelerator is worthwhile only when its extraction energy plus feature-transfer energy is below the raw-transfer and downstream-processing energy it replaces. The review must also test whether the chosen features preserve required accuracy, whether the feature format can evolve with the model, and whether keeping the detector off-chip creates a privacy or availability risk.

17.31 Figure Review: Specialization With an Energy Ledger

The diagram in Figure 17.6 makes ASIC design defensible only when profiling evidence selects a concrete specialization lane and the result is measured across more than peak speed.

The figure in Figure 17.6 routes WORKLOAD evidence through ISA, DATAPATH, MEMORY, INTERCONNECT, or CONTROL, then forces MEASURE AGAIN and a CONSEQUENCE LEDGER spanning performance, energy, area, toolchain, and verification.

The chart in Figure 17.7 shows why accelerator energy and data-movement energy must be reviewed as a single system budget.

The ILLUSTRATIVE ACCELERATOR ENERGY panel in Figure 17.7 puts H.265 and HOG near the same nJ-per-pixel scale, while the diagram’s SENSOR-ADJACENT FEATURES path shrinks 100 MB/s of raw pixels to a 5 MB/s · 20× smaller feature stream.

17.32 Summary

Hardware optimization is evidence-led platform design:

1. Start with the measured workload and target requirement.
2. Identify whether the limiter is compute, data movement, memory, wake behavior, or background power.
3. Prefer the least specialized hardware path that can move the bottleneck.
4. Treat peripherals, DMA, clock gating, and power domains as first-class optimization levers.
5. Specialize hardware only when workload stability, volume, risk, and validation evidence support it.
6. Remeasure the whole device after every hardware change.

17.33 What’s Next

17.38 Key Takeaway

Hardware optimization means selecting the MCU, peripherals, sensors, radios, accelerators, and power domains that match the workload. Idle leakage and wake cost can matter as much as peak speed.