Late in a project, physical design reports that a dozen long routes between blocks need extra pipeline stages to close timing. In a design whose blocks count cycles ("the response arrives exactly four cycles after the request"), every added stage breaks a block's assumptions, and each fix needs a design change and new verification. In a design built on valid/ready everywhere, the same request is routine: add register slices, rerun timing, done.
What property lets you add cycles of latency anywhere without changing what a system computes?
The obvious way to design blocks is to rely on known latencies: a block sends a request and samples the answer a fixed number of cycles later. It is efficient, and it couples every block to the exact timing of every path it uses. Change one path's latency, by adding a pipeline stage or moving a block, and the result arrives in the wrong cycle. The behaviour of the system depends on the latency of its wires, which is exactly the thing physical design must be free to change.
A latency-insensitive system is one whose results depend only on the sequence of values each block receives, never on when they arrive. Each block waits for valid data before it acts and holds its output until it is accepted, which is the valid/ready contract of this part. Then any channel can take any number of cycles, and the system computes the same results; only the latency and, possibly, the throughput change.
The idea has a formal basis. Carloni, McMillan and Sangiovanni-Vincentelli's theory of latency-insensitive design (2001) showed how a synchronous system can be made to tolerate arbitrary added latency on its channels. Cortadella, Kishinevsky and colleagues' work on synchronous elastic circuits (the SELF protocol) gave a simple valid/stop handshake, close to valid/ready, with proofs that networks of such elastic blocks compose: connecting correct elastic blocks gives a correct elastic system.
The model sends 0, 1, 2, ... through 0, 1, 3 and 6 full-throughput elastic stages into a sink that stalls on a fixed pseudo-random 30% of cycles after a short ready start. It records when the first word arrives, how long 30 words take, and checks that the sequence the sink receives is identical every time.
"""Latency-insensitive design: add pipeline stages, change nothing but latency.
A source sends 0, 1, 2, ... through N elastic stages (one-entry valid/ready stages
that can accept a word in the same cycle they pass one on) to a sink that is ready
for the first 10 cycles and then on a fixed pseudo-random 70% of cycles. Whatever N is, the sink must see exactly the
same sequence of values; only the time of the first word changes.
"""
import random
def run(n_stages, words=30, seed=5):
rng = random.Random(seed)
sink_ready = [True] * 10 + [rng.random() < 0.7 for _ in range(10_000)]
stages = [None] * n_stages # each holds one word or None
nxt, out, first_at, t = 0, [], None, 0
while len(out) < words:
rdy_sink = sink_ready[t]
# ready flows backwards: a stage can take a word if it is empty or emptying
ready = [False] * (n_stages + 1)
ready[n_stages] = rdy_sink
for i in range(n_stages - 1, -1, -1):
ready[i] = stages[i] is None or ready[i + 1]
# the clock edge: words move forward wherever the next place is ready
if n_stages == 0:
if rdy_sink:
out.append(nxt); nxt += 1
else:
if stages[-1] is not None and rdy_sink:
out.append(stages[-1]); stages[-1] = None
for i in range(n_stages - 1, 0, -1):
if stages[i - 1] is not None and stages[i] is None:
stages[i], stages[i - 1] = stages[i - 1], None
if ready[0] and stages[0] is None:
stages[0] = nxt; nxt += 1
if out and first_at is None:
first_at = t
t += 1
return out, first_at, t
print("stages first word at cycle cycles for 30 words same values?")
reference = None
for n in (0, 1, 3, 6):
out, first, total = run(n)
reference = reference or out
same = out == reference == list(range(30))
assert same
print(f"{n:6d} {first:19d} {total:19d} {same}")
print("PASS: every pipeline depth delivers the same values in the same order; only latency changes")It prints:
stages first word at cycle cycles for 30 words same values?
0 0 43 True
1 1 45 True
3 3 47 True
6 6 53 True
PASS: every pipeline depth delivers the same values in the same order; only latency changesThe first word arrives exactly one cycle later per stage, and the 30 values arrive identical and in order at every depth. The total time grows a little more than the added latency because the sink's stalls fall on different words when everything is shifted, but nothing about what is delivered changes. That is the property the late timing fix needs.
Latency always grows with added stages; latency insensitivity makes that safe, not free.
Throughput is preserved only if every stage can accept a word while passing one on (lesson 3.10), and only where there is no loop. Around a feedback loop, a channel whose result is needed before the next item can start, every added stage lowers throughput: a loop with k items in flight and n cycles of latency runs at no more than k/n items per cycle. This is the credit loop of Part 4 in another form.
Blocks must truly wait for data. A block that "knows" an answer comes back in four cycles and samples it then breaks the property, whatever handshake its ports have.
Approach | What it buys | What it costs |
|---|---|---|
Fixed-latency interfaces | Minimum logic and latency | Every timing change is a design change |
Latency-insensitive (elastic) interfaces | Paths can be re-pipelined late without changing behaviour | A handshake per channel; care with throughput around loops |
Elastic plus credits on long or looping paths | Throughput preserved across long round trips | Buffers sized to the round trip (lesson 4.5) |
Where | How |
|---|---|
On-chip interfaces such as AXI | Every channel uses VALID/READY with rules that never depend on a fixed latency, so interconnects can add register slices on any channel without changing behaviour. |
PCI Express and other links | Nothing assumes a fixed link latency: requests are matched to completions by tags (Part 6) and flow control uses credits, so links of any length behave the same, only slower. |
A block on an elastic interconnect has valid/ready ports but, inside, samples a read response a fixed number of cycles after issuing the request, because "the memory always answers in six cycles" (illustrative). Months later, physical design adds two register slices on the response path. The block now reads stale data twice in every few thousand requests, when its fixed-cycle sample happens to catch an old value still sitting on the bus. The handshake ports made the block look elastic; its internal assumption made it not. The fix was to act only when the response's valid is seen, and to add a check in its testbench that randomises response latency.
Each elastic stage holds a word for one cycle on its way through, so N stages add N cycles before the first word reaches the sink.
2/10 = 0.2 items per cycle. With two more stages the round trip is 12 cycles: 2/12 ≈ 0.17. More items in flight (more buffering or credits) would restore it.
If it internally assumes a fixed latency (for example sampling a response a set number of cycles after a request), changing a path's latency still breaks it.
A latency-insensitive system's results depend on the sequence of values, not on when they arrive, so stages can be added anywhere.
The valid/ready contract provides it if every block truly waits for data; theory (Carloni et al., SELF) shows such blocks compose.
Latency always grows; throughput holds on straight paths but drops around loops unless more items are kept in flight.