โ† Back to the chapter

Technical detail ยท Simulation and Validation: Evidence Strategy

Why evidence levels disagree

30.8 Why Evidence Levels Fail

A unit failure may expose a code error; an emulator failure may expose boot or timing assumptions. A HIL failure may involve voltage, a sensor, the fixture, or the firmware image. A field failure may involve the site, backhaul, or staff workflow.

Record the context. A code test needs the commit and inputs. HIL needs the board, wiring, image, and fixture setup. A field result needs the site, rollout version, and event time. These details help identify what changed when a result no longer holds.

Compare what the model and bench observed. The model may omit a fault, while the bench may have a loose cable or wrong image. Inspect packet traces, boot reason, queue state, and current. Repeat the case after fixing the suspected cause.

  • State evidence: Firmware state, queue depth, retry timer, watchdog status, update version, and boot reason.
  • Interface evidence: MQTT/CoAP exchange, TCP reset, TLS handshake, DNS failure, UART/SPI/I2C trace, and HIL input script.
  • Operational evidence: Field logs, incident timeline, support action, monitoring alert, rollback trigger, and residual-risk owner.

โ† Back to the chapter