Analytics & ML · Study deck
Cloud Data: Platforms and Services
Picture freezer readings arriving from one hundred shops.
Data Dora is your guide for this deck.
After studying this chapter
Learning objectives
You will be able to:
- Explain: The actual stack includes identity integration, device provisioning, broker limits, stream processing, serverless or container workloads, storage tiers, analytics query engines, dashboards, observability, support, and the cost of the team operating those parts.
- Explain: Google Cloud architectures often use Pub/Sub, Dataflow, Cloud Run, BigQuery, Cloud Storage, Looker, and Vertex AI, with device connectivity supplied through MQTT brokers, gateways, or partner services.
- Explain: IoT systems pay for messages, broker throughput, stream processing, serverless duration, always-on containers, databases, object storage, data transfer, warehouse scans, observability, backups, support, and engineering time.
- Explain: Autoscaling helps bursty loads.
Major section
Start With the Story
The cloud path must accept each reading, keep its time and source, store it, run a rule, send an alert, and leave evidence for support.
- Cheap intake can lead to costly storage, queries, transfer, and support.
- A working dashboard can still hide missing events.
- The result should show what was kept, lost, repeated, or acted on late.
Major section
Cloud IoT as Data Plane
The control plane manages identities, certificates, device registry records, firmware status, policies, and fleet operations.
- The data plane ingests telemetry, routes events, stores evidence, runs stream and batch jobs, feeds dashboards, and sends selected data to analytics or machine-learning systems.
- The service model changes who operates each layer.
Major section
Cloud IoT as Data Plane (continued)
Storage Hot time-series stores, lakehouse tables, object storage, warehouse tables, lifecycle rules, and archives.
- With SaaS, the team buys a complete application and accepts less control over the pipeline.
- Moving right generally transfers more platform operation to a provider, but it does not transfer accountability for device semantics, retention, access, or exit.
- Device contracts, schemas, access policy, pipeline code, retention, cost controls, and application logic.
- Standard asset monitoring or operations workflows where customization is limited.
Major section
Compare Full Service Chain
Real platforms differ most in surrounding services and team fit.
- Cloud platform selection should compare the whole data chain, not only the published message price for an IoT hub.
- AWS IoT Core often pairs with AWS IoT Greengrass, Kinesis, Lambda, Timestream, S3, Glue, Athena, Redshift, and SageMaker.
- design reading: The message charge is only one line item.
Major section
Compare Full Service Chain (continued)
The actual stack includes identity integration, device provisioning, broker limits, stream processing, serverless or container workloads, storage tiers, analytics query engines, dashboards, observability, support, and the cost of the team operating those parts.
- Azure IoT Hub often pairs with Azure IoT Edge, Event Hubs, Stream Analytics, Data Explorer, Blob Storage, Synapse, Fabric, Power BI, and Microsoft Entra ID.
- Google Cloud architectures often use Pub/Sub, Dataflow, Cloud Run, BigQuery, Cloud Storage, Looker, and Vertex AI, with device connectivity supplied through MQTT brokers, gateways, or partner services.
- Broker limits, partition plan, schema registry, dead-letter queue, and replay test.
Major section
Compare Full Service Chain (continued)
example message charge: if the contracted rate is $0.80 per million messages, 374.4 million * $0.80 = $299.52/month for messaging.
- The platform choice must also account for identity integration, stream processing, map or route services, dashboards, storage lifecycle, monitoring, support, and operating effort.
- A low message price can lose to a better-integrated platform if identity, dashboarding, data export, compliance, or operations work becomes expensive.
- Keeping all raw data in a fast tier because it is easier on day one.
Major section
Cloud Cost as Operations
Cloud cost is not a single billable unit.
- IoT systems pay for messages, broker throughput, stream processing, serverless duration, always-on containers, databases, object storage, data transfer, warehouse scans, observability, backups, support, and engineering time.
- Latency and sovereignty can be more important than cost.
- A cloud round trip may be too slow for machine protection or local control.
Major section
Cloud Cost as Operations (continued)
Some data may be legally or contractually required to stay in a region or site.
- In those cases, edge processing, regional routing, private networking, or hybrid storage are platform requirements rather than optimization extras.
- Autoscaling helps bursty loads.
- The right platform must expose enough metrics and controls to prove which pattern is true for the workload.
Major section
Cloud Cost as Operations (continued)
Lifecycle Responsibility Retention, downsampling, archive restore, deletion, and legal hold rules must be explicit and tested.
- Observability Responsibility Message drops, throttling, dead-letter growth, stream lag, late events, query scans, and cost anomalies need owners.
- Portability Responsibility Use standard MQTT topics, schema registries, exportable table formats, and documented replay paths where exit risk matters.
- When a platform review is complete, the output should be more than a selected vendor.
Deck summary
Key takeaways
The cloud path must accept each reading, keep its time and source, store it, run a rule, send an alert, and leave evidence for support.
- The control plane manages identities, certificates, device registry records, firmware status, policies, and fleet operations.
- Storage Hot time-series stores, lakehouse tables, object storage, warehouse tables, lifecycle rules, and archives.
- Real platforms differ most in surrounding services and team fit.
- The actual stack includes identity integration, device provisioning, broker limits, stream processing, serverless or container workloads, storage tiers, analytics query engines, dashboards, observability, support, and the cost of the team operating those parts.
Retrieval practice
Recall check 1 of 3
Data Dora says: answer from memory, then check your reasoning.
Q1A three-person team has 500 devices and needs a custom IoT analytics prototype in two months. Which service model is usually the best starting point?
Show answer
Answer: A PaaS often offers the best tradeoff for small teams that need a custom system quickly without operating every infrastructure layer.
Retrieval practice
Recall check 2 of 3
Data Dora says: answer from memory, then check your reasoning.
Q2The fleet example produces 374.4 million messages/month and 74.88 GB/month of telemetry. What is the best way to use those numbers when selecting a cloud platform?
Show answer
Answer: B Device and data rates should feed a full-chain cost and operations model, not a single-service price comparison.
Retrieval practice
Recall check 3 of 3
Data Dora says: answer from memory, then check your reasoning.
Q3Which platform-review artifact best prevents cloud IoT cost and operations surprises after launch?
Show answer
Answer: C Cloud platform selection must include ownership, cost, lifecycle, observability, replay, and constraints, not only a provider name.
Print reference
Answers
Answer key.
- A · PaaS often offers the best tradeoff for small teams that need a custom system quickly without operating every infrastructure layer.
- B · Device and data rates should feed a full-chain cost and operations model, not a single-service price comparison.
- C · Cloud platform selection must include ownership, cost, lifecycle, observability, replay, and constraints, not only a provider name.