DenseChip

Computer Architecture · All levels

Routing and Flow Control

Routing and Flow Control — computer architecture for silicon teams.

On-call / interview prompt

Under stress traffic, throughput plateaus and tail latency explodes while aggregate offered load is still below theoretical capacity. What is your first routing/flow-control triage?

diagram
ARCHITECTURE ANALYSIS CHAIN

1. METRIC     — IPC, CPI, MPKI, bandwidth, latency, queue depth, stall cycles
2. HYPOTHESIS — microarch or system cause ordered by likelihood
3. EXPERIMENT — trace, PMU counter, simulation, or RTL probe
4. CHANGE      — pipeline, cache, NoC, or memory hierarchy adjustment
5. VALIDATION  — workload replay, regression suite, PPA impact

Topic overview

Choose deadlock-safe routing and credit/backpressure strategy that preserve throughput under bursty and asymmetric traffic.

Mechanism to narrate

  • Section: NoC and Interconnect Architecture

  • Primary artifact: Routing/flow-control stress verification report

  • Downstream dependency: QoS guarantees, coherency response latency, and software-visible jitter.

Staff/principal ownership model

Own Routing and Flow Control as a product architecture decision, not a page of notes. A senior architect names the metric, the mechanism, the cross-team dependency, and the smallest evidence-producing experiment.

diagram
STAFF ARCHITECTURE REVIEW MEMO — NoC and Interconnect Architecture / Routing and Flow Control

1. Current state
   - Failing / watched metric: Routing/flow-control stress verification report
   - Workload / benchmark / trace: <fill before review>
   - Model tag, RTL tag, simulator version, PMU setup: <fill before review>
   - Scope: core, cache level, NoC path, coherency domain, accelerator, or SoC budget

2. Root-cause hypothesis
   - Most likely mechanism: <name pipeline/cache/NoC/coherency/perf mechanism>
   - Competing hypothesis: <name the second plausible cause>
   - Evidence still missing: <counter, trace, waveform, model sweep, or workload slice>

3. Proposed action
   - Minimal reversible change: <microarchitecture, policy, sizing, traffic, or software contract change>
   - Expected improvement: <metric delta>
   - Regression risk: Mis-tuned flow control can collapse effective bandwidth despite nominal link capacity.

4. Regression and signoff
   - Re-run: Routing/flow-control stress verification report
   - Must not regress: QoS guarantees, coherency response latency, and software-visible jitter.
   - Decision owner: NoC microarchitecture owner

Sub-lessons in this topic

  1. mechanism — Mechanism

  2. inputs-outputs — Inputs & Outputs

  3. reports — Reports & Metrics

  4. debug-playbook — Debug Playbook

  5. worked-example — Worked Example

  6. pitfalls — Pitfalls & Red Flags

  7. interview — Interview Drills

  8. checklist — Review Checklist

  9. theory-deep-dive — Theory Deep Dive

  10. design-space — Design Space Exploration

  11. case-study-expanded — Extended Case Study

  12. step-by-step-walkthrough — Step-by-Step Walkthrough

  13. comparison-matrix — Comparison Matrix

  14. software-programmer-view — Software / Programmer View

  15. silicon-ppa-impact — Silicon & PPA Impact

Related topics

Key takeaways

  • Master Routing and Flow Control through workload metrics, not feature lists.

Architecture deep dive

NoC is a queueing system — bandwidth, latency, and deadlock are coupled.

Concept diagram

diagram
NoC TOPOLOGY SKETCH

CPU0 ──┐      ┌── LLC0 ── DRAM0
       R0 ─── R1
CPU1 ──┘      │
              R2 ─── R3 ── GPU/DMA
              │      │
             NPU    LLC1 ── DRAM1

Look for: hot links, cyclic dependencies, VC starvation, and tail latency.

Metric graph

diagram
LATENCY DISTRIBUTION

p50    ██████  32 ns
p90    ████████████  71 ns
p99    ████████████████████████  210 ns
p99.9  █████████████████████████████████  480 ns

Averages hide QoS failures.

Metrics and artifacts

  • link utilization

  • average latency by master

  • retry/backpressure counts

  • QoS violation log

Mini case study

Average latency looks fine but tail latency spikes for CPU coherent reads when GPU DMA runs. QoS and separate VCs fix the starvation without doubling link width.

Debug branches

  • If deadlock, check credit loops and routing restrictions first.

  • If latency tail long, inspect arbitration and buffer depth.

Senior review question

Ask: what single metric would prove this concept is working or failing on your workload?

Key takeaways

  • Connect every architecture claim to a workload and measurable metric.

  • State verification and PPA impact before proposing design changes.

Common pitfalls

  • Feature-driven design without MPKI/IPC/bandwidth evidence.

  • Ignoring coherency and NoC traffic in cache and accelerator sizing.

Practice this lesson