A product requirement says: "the interconnect must not corrupt data more than once in the life of the fleet". The link datasheet says: "BER 1e-12". Between those two sentences sit packet sizes, CRC widths, replay costs, link counts and years of operation. A reliability budget connects them with a few lines of arithmetic, and shows which number actually matters.

How do you get from a bit error rate to replay rates, latency impact and undetected errors?

The obvious answer, and why it breaks

The obvious reading of "BER 1e-12" is "errors are negligible". For performance that is usually true. For data integrity it is the wrong question: at 32 Gb/s, 1e-12 is an error every half-minute, every link, all day, for years. What matters is how many of those errors could get past the checks, multiplied across every link in every product over its lifetime. Different questions use the same BER very differently.

The idea: a chain of rates

• Replay rate ≈ packets per second × packet error probability: sets the bandwidth and latency cost of recovery.

• Undetected rate ≈ bad packets per second × the chance the check misses one (at most about 2⁻³² for a random error pattern under a 32-bit CRC; much lower for single errors and short bursts, which it always catches).

• Fleet total = undetected rate per link × links per product × products × years.

Build it: a budget calculator

The calculator takes a 32 Gb/s link at BER 1e-12, 280-byte packets (256 bytes of payload), a 500 ns replay round trip and a 32-bit CRC, and works through the chain.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

"""A reliability budget for one link (PCIe-like, illustrative packet sizes).

From a raw bit error rate to how often packets are replayed, what replay costs in
bandwidth and latency, and how often an error slips past the CRC undetected.
Assumes independent bit errors (real links also see bursts) and a 32-bit CRC that
lets an errored packet through with probability 2^-32 (pessimistic).
"""
RATE = 32e9          # bits per second on the link
BER = 1e-12          # raw bit error rate
PKT = 280 * 8        # bits per packet: 256-byte payload plus 24 bytes of overhead
RTT_NS = 500         # replay round trip, ns
CRC_BITS = 32

errors_per_s = RATE * BER
pkt_error = 1 - (1 - BER) ** PKT                    # probability a packet has at least one error
pkts_per_s = RATE / PKT
replays_per_s = pkts_per_s * pkt_error
lost_fraction = replays_per_s * RTT_NS * 1e-9          # each replay resends one round trip of link time
undetected_per_s = replays_per_s * 2 ** -CRC_BITS     # pessimistic: CRC-32 catches all single errors and short bursts
years = 1 / undetected_per_s / (3600 * 24 * 365)

print(f"bit errors per second:        {errors_per_s:.3f}  (one every {1 / errors_per_s:.0f} s)")
print(f"packet error probability:     {pkt_error:.2e}")
print(f"replays per second:           {replays_per_s:.3f}")
print(f"bandwidth lost to go-back-N:  {lost_fraction * 100:.2e} %  (a round trip of packets per error)")
print(f"added latency per replay:     about {RTT_NS} ns, about once every {1 / replays_per_s:.0f} s")
print(f"undetected errors (CRC-32):   one every {years:,.0f} years per link")
assert abs(errors_per_s - 0.032) < 1e-9 and years > 1000
print("PASS: at BER 1e-12 replay is rare and cheap; the undetected rate is what the CRC width must make negligible")

It prints:

bit errors per second:        0.032  (one every 31 s)
packet error probability:     2.24e-09
replays per second:           0.032
bandwidth lost to go-back-N:  1.60e-06 %  (a round trip of packets per error)
added latency per replay:     about 500 ns, about once every 31 s
undetected errors (CRC-32):   one every 4,256 years per link
PASS: at BER 1e-12 replay is rare and cheap; the undetected rate is what the CRC width must make negligible

Performance is a non-issue: one replay every half-minute, costing a few hundred nanoseconds. Integrity is where the budget is spent: with a pessimistic estimate, one undetected error per link every few thousand years. That sounds safe until it is multiplied by the fleet.

Worked numbers: the fleet

Hundreds of silent corruptions a year across the fleet, under a pessimistic model, from a link whose datasheet number sounded negligible. That is why fast links use 32-bit CRCs at minimum, add end-to-end checks (lesson 10.14), and why the realistic undetected rate (much lower, since CRC-32 catches every single-bit error and every burst up to 32 bits) is worth computing properly for the actual error statistics.

What breaks

A design (illustrative) chose a 16-bit CRC for a high-speed internal link because "BER is 1e-12, errors are negligible". The per-link undetected rate looked tiny; across a large fleet it predicted silent corruptions every few days. The budget, done before tape-out, would have pointed to a 32-bit CRC for a few extra bits per packet.

Key takeaways

• A reliability budget chains rates: errors/s = rate × BER, then replay rate, then undetected rate.

• At BER 1e-12 replay costs almost nothing; the undetected-error rate, multiplied across the fleet, is what sizes the CRC.

• Budget every link against a stated requirement, using realistic error statistics.

Related topics

• Part 10 overview: Reliability and Recovery