Cloud, SDN & Production Architectures · Study deck

Production Fleet Management

Imagine an operator who asks for an update on one hundred pumps.

Cloud Clara is your guide for this deck.

productionmanagement

Cloud Clara, the module guide, in a scene from this chapter.

After studying this chapter

Learning objectives

You will be able to:

Explain: If a gateway buffers reports during a cellular outage, the boundary must show whether the local flood-safe rule continues, which remote commands are paused, and who reviews the delayed data when the link returns.
Explain: When requested and reported state diverge, the system needs a timeout, owner, fallback behavior, escalation rule, and closure proof rather than a silent assumption that the fleet will eventually converge.
Explain: A retry storm can overload gateways, a rollback can target devices that never applied the update, and an operator can close an incident while the local fallback is still active.

Major section

Start With the Fleet State Question

The control screen says the job was sent.

A fleet needs both the state we want and the state each device reports.
Firmware is the software stored on a device.
Admit a device only when its identity is known.
The fleet will change after release.

Major section

Start With the Fleet State Question (continued)

If an update fails, keep the old working state or move the device to a known safe state.

The record must survive those changes.
Practitioner covers entry, update, support, retirement, and proof.
Under the Hood explains state drift, staged rollout, and degraded modes.
Those deeper controls refine the simple group record.

Major section

Start With the Fleet State Question (continued)

A fleet may be updated, degraded, isolated, misconfigured, or healthy, and the operator needs proof of which state is real.

They do not turn a sent request into proof of a completed change.
The owner asks for a change.
Each device reports its own state.
The rollout can pause.

Major section

Production Management Operating Model

Production management starts with the operating boundary.

The boundary names the devices, gateways, remote services, operators, users, and support teams that share responsibility for the live system.
That observation connects this visual to the chapter's running narrative: use it to justify the next design decision and to record what evidence would confirm it in operation.

Production architecture management operating model showing operating boundary, lifecycle control, identity control, update control, observability, fallback behavior, support ownership, and decision records connected as one decision loop.

Major section

Management Domains

A production architecture has multiple management domains.

The proposition under review is: Production management domains showing lifecycle, identity, update, observability, degraded operation, service behavior, support ownership, and records around the operated fleet.
This supports management domains.
Names who acts on alerts, what proof closes an issue, and how lessons from incidents update the architecture record.

Production management domains showing lifecycle, identity, update, observability, degraded operation, service behavior, support ownership, and records around the operated fleet.

Major section

Fleet Lifecycle Control

A production system should know which state each asset is in and which transitions require proof.

At fleet lifecycle control, Fleet Lifecycle Control makes the architecture testable.
Between them, the fleet lifecycle control relationship becomes visible: Fleet lifecycle control.

Major section

Update and Configuration Control

Remote change is part of the production architecture.

The management model should support small changes, large changes, pause decisions, rollback decisions, and operator communication without relying on a single all-or-nothing release.
This recovery proof, rollback path sequence supports the claim: Update and configuration control.

Major section

Common Pitfalls

An Edge-Fog-Cloud diagram is not enough.

The architecture must name who owns health, updates, identity, fallback, and support action.
A change route without validation, pause, and recovery proof can turn a small defect into a fleet issue.
Metrics and logs help only when they route to a support owner with a clear action and closure signal.

Major section

Production as Living Boundary

A production architecture is not finished when the first device sends data.

The prototype question is whether readings arrive.
The production question is harder: whether each sensor has a known asset record, credential, firmware version, policy, gateway route, support owner, and retirement path.
Good production evidence is mundane but decisive.

Major section

Production as Living Boundary (continued)

If a gateway buffers reports during a cellular outage, the boundary must show whether the local flood-safe rule continues, which remote commands are paused, and who reviews the delayed data when the link returns.

The same architecture must survive cohort updates, credential rotation, stale health reports, service dependency failures, support handoffs, and physical replacement without losing proof of ownership.
Together those artifacts show whether the live fleet is still inside the promised boundary.
If one artifact is missing, the architecture should say which owner must restore it before the fleet change proceeds.

Major section

Fleet Change Record

Every rollout, update, policy change, or retirement should leave a record that joins the technical state to the operating owner.

Without that record, production work depends on memory during the exact moments when memory is least reliable.
For a firmware rollout, include the exact cohort query, excluded devices, requested version, reported version, delivery result, post-change health window, pause threshold, rollback package, and owner.
The record should separate the platform state from the field state.

Major section

Requested vs Actual State Drift

Production systems carry several kinds of state at once.

Command state says whether an action was accepted, expired, rejected, or still pending.
These states drift for ordinary reasons: devices sleep, links fail, gateways buffer, update cohorts pause, clocks skew, identities rotate, and support tickets cross team boundaries.
A production architecture should make state mismatch visible.

Major section

Requested vs Actual State Drift (continued)

When requested and reported state diverge, the system needs a timeout, owner, fallback behavior, escalation rule, and closure proof rather than a silent assumption that the fleet will eventually converge.

The implementation detail can vary, but the invariant is the same.
A device-shadow service, MQTT retained state, gateway queue, update orchestrator, and ticket system may each hold a different piece of truth.
Without that separation, recovery actions can make incidents worse.

Major section

Requested vs Actual State Drift (continued)

The architecture should define which source is authoritative for each decision and which mismatches are allowed during sleep, outage, or rollout windows.

A retry storm can overload gateways, a rollback can target devices that never applied the update, and an operator can close an incident while the local fallback is still active.
Production reviews should therefore ask for invariants rather than screenshots.
Those invariants make different platforms comparable without pretending every vendor exposes the same controls.

Deck summary

Key takeaways

The control screen says the job was sent.

If an update fails, keep the old working state or move the device to a known safe state.
A fleet may be updated, degraded, isolated, misconfigured, or healthy, and the operator needs proof of which state is real.
Production management starts with the operating boundary.
A production architecture has multiple management domains.

Retrieval practice

Recall check 1 of 3

Cloud Clara says: answer from memory, then check your reasoning.

Q1A staged firmware rollout dashboard says 96% complete. The requested version is 2.3, some devices still report 2.2, several gateways have queued commands, and one region has stale health reports. What should production management check first?

ACompare requested, reported, queued, buffered, freshness, pause, and rollback states.

BAccept the rollout because most devices show 2.3 and the dashboard is nearly complete.

CRetry pending commands immediately so the dashboard number reaches 100 percent.

DIgnore stale health reports until support tickets prove users are affected.

Show answer

Answer: A Fleet management distinguishes requested state, reported state, command state, gateway queue state, health freshness, and pause or rollback criteria before accepting a rollout.

Retrieval practice

Recall check 2 of 3

Cloud Clara says: answer from memory, then check your reasoning.

Q2A regional stormwater pilot reports pump current and sump level from three stations through local gateways. The next rollout adds 40 stations, shared maintenance crews, staged firmware updates, and local flood-safe behavior during backhaul outages. Before approval, which missing proof matters most?

AExpanded dashboard screenshots from the pilot plus a manual inspection promise, because visible data and human checks prove rollout readiness.

BLifecycle states, identity owners, update pause or rollback, stale-health signals, local safety, and alert owners.

CProtocol diagrams and pump-current charts only, because lifecycle, support, and rollback rules can wait until after launch.

DMore cloud storage and wider dashboard access, because capacity and visibility replace local outage behavior and support ownership.

Show answer

Answer: B A technically successful pilot is not enough for production sign-off.

Retrieval practice

Recall check 3 of 3

Cloud Clara says: answer from memory, then check your reasoning.

Q3A fleet dashboard shows that an update was requested for 500 devices, but 80 sleeping devices have not reported the new version and some gateways are buffering. What is the production-management response?

ATrack requested, reported, command, buffer, and support states before recovery action.

BUse the successful 500-device request as the completed-rollout milestone and reconcile versions later.

CWait for gateway queues to drain before evaluating the missing device reports.

DWait for users to report problems before assigning owners, because sleeping devices normally delay fleet-wide state convergence.

Show answer

Answer: A Production management separates requested, reported, command, buffer, and support state so rollout drift is visible and tied to recovery ownership.

Print reference

Answers

Answer key.

A · Fleet management distinguishes requested state, reported state, command state, gateway queue state, health freshness, and pause or rollback criteria before accepting a rollout.
B · A technically successful pilot is not enough for production sign-off.
A · Production management separates requested, reported, command, buffer, and support state so rollout drift is visible and tied to recovery ownership.