A cloud provider rents out a server with an accelerator card. The tenant's data travels between the processor and the card over a PCI Express link on the motherboard. Anyone with physical access to the machine, or control of a switch on the path, could attach an interposer to that link: read every packet, change a write's data and fix up its CRC, or record a packet and play it again later. The link's error detection would notice none of it.
How is a link protected against someone who can read and change what is on it?
The obvious answer is that links already have CRCs (lesson 10.4). A CRC is designed against random errors; it has no secret. Anyone who changes a packet can recompute the CRC, and the receiver accepts the result. Sequence numbers stop accidental duplicates, but an attacker can replay a packet with a valid number in a later session. Protection against an adversary needs secrets: keys that the attacker does not have.
Protection | Defends against | Mechanism |
|---|---|---|
Confidentiality | Reading the data | Encrypt the payload with a key shared by the two ends |
Integrity and authenticity | Changing packets, or injecting new ones | A cryptographic tag (MAC) computed with the key; the receiver rejects packets whose tag does not verify |
Replay protection | Recording and resending valid packets | A counter in each packet, covered by the tag; the receiver rejects counters it has already seen |
Authenticated encryption, such as AES-GCM, gives the first two at once: it encrypts the payload and produces a tag over the payload and the unencrypted header fields. Keys are established by a separate key-exchange protocol between the two ends and changed periodically.
Bandwidth: each packet (or group of packets) carries a tag, often 96 or 128 bits, and a counter.
Latency: encryption and tag checking add pipeline stages at each end; a packet cannot be used until its tag verifies.
Ordering and replay interaction: link-level replay (lesson 10.11) resends packets that must not be mistaken for an attacker's replay; counters and replay buffers must be designed together.
Key management: keys are set up, refreshed and revoked through a separate protocol, and must survive link retraining and resets correctly.
Scope: hop-by-hop protection decrypts at every switch, which must then be trusted; end-to-end protection passes through switches untouched.
AES-GCM takes a key, a nonce (its initialisation vector, usually 96 bits) and the data. Its central rule: a nonce must never be used twice with the same key (NIST SP 800-38D). If two different packets are ever encrypted under one key with one nonce, an attacker who records both can XOR the ciphertexts to get the XOR of the two plaintexts, and can recover the value that lets them forge valid tags. The tag no longer protects anything.
Link designs usually build the nonce from the packet counter, so one counter serves two purposes with different rules:
Purpose | Who enforces it | The rule |
|---|---|---|
Replay detection | The receiver | Reject a packet whose counter it has already accepted, or that is outside its window |
Nonce uniqueness | The sender | Never encrypt different data with a counter value already used under this key, in this direction |
Reset: if the link resets and both ends restart the counter at 0 but keep the key, the first packet after reset reuses nonce 0 with new data. The receiver's replay check does not notice, because it was reset too. Treat a reset (or any loss of counter state) as the end of the key's use: agree a fresh key, or a fresh key-derivation input, before traffic resumes.
Wrap: a counter that wraps repeats every nonce. Rekey well before the counter can wrap; a sender that reaches the limit must stop rather than reuse.
Direction and streams: each direction, and each independent stream, needs its own key or its own fixed field in the nonce, so the two ends never pick the same value.
Link-level retransmission: a link replay (lesson 10.11) resends the same ciphertext with the same nonce, which reveals nothing new. The link layer discards its own duplicates by sequence number before the integrity check, so the receiver's cryptographic replay check never sees them. Re-encrypting retransmitted data under a new counter is also safe; re-encrypting different data under an old counter never is.
Protocol | Link protection |
|---|---|
PCI Express IDE | Integrity and Data Encryption: AES-GCM on TLPs, per link or end to end through switches (fact P35) |
Ethernet MACsec | IEEE 802.1AE: AES-GCM on frames hop by hop, with keys agreed by MKA (fact S20) |
On-chip interconnects | Usually none: the chip boundary is the trust boundary |
A design (illustrative) added encryption to a chip-to-chip link but left the packet header, including the target address, outside the integrity tag to save latency. An attacker on the link could not read the data but could change the address of an encrypted write, redirecting it to a different location. Covering the header with the tag (authenticating it without encrypting it) closed the hole.
Otherwise an attacker can change the address or type of an encrypted packet without detection.
The first packets after reset reuse nonces already used before the reset, with different data: two ciphertexts under the same key and nonce leak the XOR of their plaintexts and allow tag forgery. The replay check does not help, because the receiver's counter was reset too. Establish a new key (or new key-derivation input) on every reset, before traffic resumes.
Every switch or intermediate hop, which decrypts and re-encrypts the traffic.
CRCs defend against noise, not attackers: anyone can recompute them.
Adversarial protection needs keys: encryption, a cryptographic integrity tag over the whole packet, and replay counters.
It costs bandwidth, latency and key management, and must be designed with retry and reset: a GCM nonce must never repeat under one key, so rekey on reset and before the counter wraps.