A processor reads a word from slow flash, then a word from fast SRAM. The SRAM answers in 5 ns; the flash in 500 ns. If responses must come back in request order, the SRAM's answer sits in a buffer for 495 ns waiting for the flash. The SRAM is fast, and every read that follows a flash read in the same order is slow anyway.
Must responses return in request order, and who pays for it either way?
The obvious rule is in-order completion: responses come back in the order requests were sent. Requesters love it, because the next response always belongs to the oldest request and no tags are needed. But someone must enforce it: either every response waits for all earlier ones, or the fabric holds early responses back. With targets of different speeds, in-order completion makes every request as slow as the slowest one ahead of it.
Policy | Who pays | Cost |
|---|---|---|
In order, everything | The fabric or the target holds early responses | Fast responses wait for slow ones (head-of-line blocking, lesson 4.14); buffers in the fabric |
Out of order, everything | The requester | Tags, and a reorder buffer if the requester needs order (lesson 6.6) |
In order per ID | Shared: the fabric orders within an ID; the requester chooses IDs | Requests that must stay ordered share an ID; independent ones use different IDs |
Ordering per ID gives requesters a dial. Use one ID for everything and you get full ordering. Use a different ID for each independent stream (each target, each thread, each buffer) and those streams no longer wait for each other.
Illustrative numbers: alternate reads to flash (500 ns) and SRAM (5 ns), 10 reads outstanding.
In order: each SRAM read waits for the flash read ahead of it -> ~500 ns each
Out of order (different IDs): SRAM reads complete in ~5 ns; flash reads in ~500 ns
Average latency: in order ~500 ns; out of order (500 + 5) / 2 = 252.5 ns
SRAM throughput: in order limited by the flash; out of order limited only by the SRAM and its tagsTags or IDs to match every response (lesson 6.4).
Order where the program needs it: a processor that must retire loads in program order puts responses in order itself, in a reorder buffer.
Same-address hazards: a read and a write to the same address on different IDs have no order between them; the requester must wait for the write's response before the read if it needs to see the new value (Part 7).
Protocol | Completion order |
|---|---|
AXI | Same ID: in order. Different IDs: any order. Read data of different IDs may even be interleaved beat by beat (lesson 6.7). |
PCI Express | Completions for different requests may return in any order; each carries its tag. The pieces of one split read return in address order (lesson 6.8). |
APB | Only one transfer at a time, so the question never arises. |
An accelerator (illustrative) used a single AXI ID for all its reads, because the first version only read from one memory. When a second, slower memory was added, every read to the fast memory queued behind reads to the slow one: same ID, so the interconnect kept them in order. Giving each memory its own ID halved the accelerator's average read latency without changing any hardware outside it.
In request order: same ID means in order, so the fast target's data waits for the slow one's.
Tags to identify responses and a reorder buffer to hold early ones until all earlier responses have arrived.
In-order completion makes fast responses wait for slow ones; somebody must buffer them.
Out-of-order completion moves the cost to the requester: tags and, if needed, a reorder buffer.
Ordering per ID lets requesters keep order only where it matters.