An architect hands you a one-paragraph brief and asks for an interface specification by next week. Nobody will tell you which handshake to use or how many outstanding transactions to allow; those are your decisions, and you will have to defend each one in review. This capstone is that week, compressed.
Given only requirements, can you design an interface and justify every choice?
Brief | Requirements |
|---|---|
A. Accelerator link (on die) | An accelerator reads and writes 64-byte lines from the memory system at 64 GB/s sustained. Memory latency is 100 ns. Clock 1 GHz. Several requesters inside the accelerator. Must not deadlock with the memory system's other traffic. |
B. Sensor bus (board) | Twenty low-rate sensors on one board, each sampled 100 times per second, 16 bytes per sample. As few pins as possible. Sensors from several vendors. A failed sensor must not take down the bus. |
C. Die-to-die link (package) | Two dies in one package exchange 256 GB/s in each direction, with lowest latency. Raw bit error rate about 1e-15. Both dies from the same company, but designed by different teams. |
Pick one. Answer the twenty questions for your design, write numbered rules, size every buffer and counter with a calculation, and list what you deliberately leave out.
Sizing. Start from Little's law (lesson 6.2):
Line rate needed: 64 GB/s / 64 B per line = 1.0e9 lines per second
Outstanding reads: 1.0e9 /s x 100 ns = 100 lines in flight
Tag width: 100 outstanding -> 7-bit tags (128)
Per-hop credit loop: 8-cycle round trip x 1 flit per cycle = 8 credits per hop
Data width: 64 GB/s at 1 GHz = 64 bytes per cycle (512 bits)Question | Choice | Why |
|---|---|---|
Timing | Common clock; one crossing at the memory system boundary | On die (lesson 1.1) |
Handshake | Valid/ready on each hop | Register slices can be inserted freely (lesson 3.8) |
Flow control | Valid/ready on die; credits only if a hop's round trip grows | Short wires; ready is simplest (lesson 4.15) |
Transactions | Line-sized reads and writes, posted writes with a completion count for fences | Line traffic; fences need to know writes finished |
Concurrency | 128 tags; out-of-order completion | Little's law: 100 outstanding |
Ordering | Unordered by default; same-tag in order; explicit fence | Throughput; software orders only where needed (lesson 7.5) |
Sharing | Round-robin among the accelerator's requesters | Fairness without starvation |
Deadlock | Separate request and response channels with independent back-pressure; responses always accepted (pre-allocated by tag) | Breaks the request/response cycle (lesson 9.4) |
Errors | Error response per transaction; ECC on data from memory, poison passed through | On-chip wires are reliable; memory errors are not (lesson 10.8) |
Bring-up | Reset rules; requests only after a ready indication from the memory system | Order-independent start (lesson 11.2) |
Power | Request/accept/deny handshake; accept only with no tags outstanding | Lesson 12.4 |
Coherence | I/O coherent reads and writes through the memory system | Shares data with CPUs without cache maintenance (lesson 13.7) |
Security | Requests carry stream ID and non-secure bit; IOMMU on the path | Lessons 14.1, 14.3 |
Verification | Checker per hop, per-tag scoreboard, formal on tag allocation and fences | Part 15 |
Left out deliberately: link-level retry (reliable wires), credits (unless a hop is long), bursts beyond one line (line traffic only).
Good: every number has a calculation; the deadlock argument (responses always accepted because buffer space is pre-allocated per tag) is explicit.
Question: 100 outstanding assumes the latency never exceeds 100 ns. Under contention it will; with 128 tags the link keeps full rate up to 128 ns, beyond which throughput drops proportionally. State that.
Question: what does the fence wait for: write responses from memory, or acceptance by the memory system? Early acknowledgement changes what software can assume (lesson 7.4).
Missing: a timeout for lost responses, and what a tag does after an error.
Total rate is 20 x 100 x 16 B = 32 kB/s, so an I2C-like two-wire, open-drain bus with addresses suffices (lesson 16.4); multi-vendor favours a standard; per-byte acknowledgement detects absent sensors; a stuck-bus recovery and per-transaction timeout stop one failed sensor from hanging the bus.
Very wide, short-reach parallel lanes with a forwarded clock (lesson 16.14); flits; credits because the round trip is many cycles at 256 GB/s; a CRC with link-level retry even at 1e-15, because the fleet-wide rate of errors is not zero (lesson 10.18); capability exchange at start-up since two teams build the ends.
Design from requirements: size with Little's law, round trips and error rates before choosing mechanisms.
Answer all twenty questions, write numbered rules, and list what you leave out and why.
State the assumptions behind each number and the behaviour when they break.