NoSec
No DTLS. Plain coap:// on 5683. Security must come from the link or network layer.
16 CoAP DTLS/OSCORE Security
coap
security
dtls
oscore
16.1 Start With the Proxy Boundary
A CoAP proxy can save energy and bridge protocols, but it also raises a blunt question: who can see plaintext? DTLS protects a hop; OSCORE can protect the object across a proxy; credentials and replay windows decide whether the protection is auditable.
Use this page as the security boundary map. Each mode is useful only when the review can say what is hidden, what remains visible, where trust terminates, and what evidence proves replay protection.
16.2 Learning Objectives
After this page, you should be able to:
- Compare NoSec, PreSharedKey, RawPublicKey, Certificate, and OSCORE security modes for constrained CoAP deployments.
- Explain what DTLS protects on a CoAP hop and where that protection terminates.
- Describe why OSCORE can preserve end-to-end object security across CoAP proxies.
- Specify credential, handshake, session, replay, and proxy-trust evidence for a CoAP security review.
16.3 Why This Contract Comes After Security and Applications
CoAP Security and Applications covers DTLS, credentials, constrained deployments, and real application use cases. This page narrows in on the deeper security contract: which protection layer is in use, what metadata it hides, whether a proxy can see plaintext, and which replay and credential records make the deployment auditable.
The CoAP security lesson remains tied to deployment choices: CoAP security controls such as DTLS, OSCORE, credentials, proxies, replay windows, and audit records must line up with the threat model.
Use it when a CoAP deployment includes healthcare, building control, industrial monitoring, caching proxies, HTTP translation, multicast, or sleepy devices that cannot pay a full handshake on every wake cycle.
Overview: Four Security Modes, and What DTLS Actually Protects
RFC 7252 defines four security modes for CoAP. NoSec means no protocol-layer security: plain coap:// on UDP port 5683, relying on something else (link-layer keys, a physically closed network) for protection. The other three all run DTLS and are addressed as coaps:// on UDP port 5684: PreSharedKey uses symmetric keys shared in advance, RawPublicKey uses an asymmetric key pair with no certificate, and Certificate uses an X.509 chain validated against a trust anchor.
What matters most is the scope of protection. DTLS secures the entire CoAP message between the two DTLS endpoints — the method, the URI path, every option, and the payload. That is why a path like /patient/12345/vitals is concealed under DTLS, whereas encrypting only the payload would leave that revealing metadata in the clear. The trade is a handshake: several messages to agree keys before any request flows.
For a closed hospital ward, a monitor might be provisioned with identity bed-042 and a 128-bit PSK, then send coaps://gateway/patient/12345/vitals over a long-lived DTLS session. A passive sniffer can still see that UDP packets are going to the gateway on port 5684, but cannot read the CoAP code, Uri-Path, Observe option, content format, or vital-sign payload. For a third-party device ecosystem, certificates may be worth the larger handshake because the server can validate vendor identity without manually sharing a secret with every device.
PreSharedKey
DTLS with symmetric keys shared in advance. Cheapest handshake; ideal for closed deployments.
RawPublicKey
DTLS with an asymmetric key pair and no certificate authority — asymmetric strength without PKI.
Certificate
DTLS with an X.509 chain validated to a trust anchor — full PKI for open ecosystems.
Practitioner: DTLS Is Point-to-Point and Terminates at Proxies
Choosing a DTLS mode is a resource decision. PreSharedKey has the lightest handshake and no certificate parsing, so it suits tightly managed fleets. RawPublicKey gives asymmetric-key strength without the cost and operational burden of a certificate authority. Certificate mode fits open ecosystems where devices from many vendors must be validated to a common trust anchor. To keep the handshake from dominating energy use, keep the DTLS session open and reuse it across many requests, and rely on the handshake's HelloVerifyRequest cookie to blunt spoofed-source denial-of-service before the server commits any state.
A sizing example makes the trade visible. If a meter sends one reading every 30 seconds, it sends 2,880 CoAP reports per day. A PSK DTLS handshake of roughly 500-700 bytes is expensive if repeated for every report, but almost negligible if the device keeps one session alive for an hour or a day and pays only the DTLS record overhead on each message. A deep-sleep sensor is different: if RAM is lost on every wake, the DTLS session state and sequence numbers vanish, so either session resumption state must be stored safely or OSCORE/EDHOC-style object security should be considered.
The architectural catch: DTLS secures exactly one hop between two endpoints. A CoAP proxy that caches responses or translates CoAP to HTTP must terminate the DTLS session, decrypt, and re-encrypt on the far side — which means the proxy sees plaintext. If you do not fully trust that middlebox, DTLS alone does not give you end-to-end confidentiality.
Mode
Credential
Needs a CA/PKI?
Best fit
PreSharedKey
Shared symmetric key
No
Closed fleet with pre-provisioned keys.
RawPublicKey
Bare key pair
No
Asymmetric security without certificate operations.
Certificate
X.509 chain
Yes
Open, multi-vendor ecosystems needing trust anchors.
OSCORE
Derived AEAD keys
No (uses a shared master secret)
End-to-end protection across untrusted proxies.
Under the Hood: OSCORE Protects the Message Object End-to-End
OSCORE — Object Security for Constrained RESTful Environments (RFC 8613) — answers the proxy problem by securing the CoAP message rather than the transport. It uses COSE (CBOR Object Signing and Encryption) with an AEAD algorithm (default AES-CCM-16-64-128) and derives its keys with HKDF-SHA-256 from a shared Master Secret. The request method, the payload, and the sensitive options are encrypted and integrity-protected inside a compact COSE object, which is carried as the payload of an outer CoAP message that bears the OSCORE option (option 9) and a generic outer code such as POST.
DTLS protects the hop; OSCORE protects the object. A proxy can terminate DTLS and still see plaintext, but an OSCORE-protected request stays encrypted between the original client and the final server while the proxy reads only the outer routing fields it needs.
The clever part is which options stay visible. OSCORE classifies options into an encrypted inner class and an unprotected outer class: options a proxy legitimately needs to do its job — Uri-Host, Proxy-Uri, Observe, and the block options — remain in the outer, readable header, while content-bearing options like Uri-Path and Uri-Query are encrypted inside. So a forwarding or caching proxy can still route and cache the outer message, yet it cannot read the protected content. Replay protection comes from a per-sender sequence number used to build the AEAD nonce, checked against the recipient's replay window. Because protection lives in the object, OSCORE survives untrusted proxies and can even run over a DTLS hop for defense in depth.
COSE object
An AEAD-protected CBOR structure holding the encrypted method, payload, and inner options.
OSCORE option (9)
Marks a message as OSCORE-protected and carries the key ID and partial IV a recipient needs.
Inner vs outer options
Routing/caching options stay outer and visible; content options are encrypted inner, so proxies still work.
Sequence-number replay guard
A per-sender counter forms the AEAD nonce and is checked against a replay window.
16.4 Release Checklist
Before shipping a CoAP security contract, verify these records:
- The chosen security mode is explicit: NoSec, PreSharedKey, RawPublicKey, Certificate, OSCORE, or DTLS plus OSCORE defense in depth.
- Credential provisioning names identity format, key source, trust anchor or master secret, rotation path, and revocation or decommissioning behavior.
- Handshake and session policy says whether DTLS sessions are reused, resumed after sleep, or replaced by object security for deep-sleep devices.
- Proxy and gateway diagrams mark where DTLS terminates, which components see plaintext, and whether OSCORE is required for end-to-end confidentiality.
- Replay protection records sender sequence numbers, replay-window handling, reset behavior, and incident evidence for dropped or repeated protected messages.
16.5 See Also
- CoAP Security and Applications for deployment scenarios and application patterns.
- CoAP Message Format for tokens, options, and message structure.
- CoAP Observe Extension for long-lived notifications that interact with sessions and replay windows.
- Cryptography for IoT for AES and symmetric-key foundations behind DTLS-PSK and OSCORE AEAD.
16.6 Next
Return to CoAP Security and Applications, then continue to CoAP Implementation Labs for operational practice.