NoC Topology Tradeoffs — Extended Case Study

Extended case study

A review is called because a workload regresses after a NoC Topology Tradeoffs change.

Background

A stable baseline existed until a NoC and Interconnect Architecture change improved one benchmark and regressed a product workload on NoC topology comparison dashboard.

Symptoms observed

• Regression in NoC topology comparison dashboard

• Sim vs silicon disagreement

• Pressure to revert or ship risk

Investigation timeline

1. Freeze tags

2. Reproduce

3. Cluster

4. Experiment

5. Validate

6. Memo

Root cause

A hidden assumption in NoC Topology Tradeoffs failed under an unrepresented workload phase.

Fix and validation

• Reproduce hotspot with deterministic traffic seed and trace window.

• Overlay hotspot links on floorplan-aware route lengths.

• Compare offered load versus achieved throughput per region.

• Test one topology-aware remap or clustering intervention.

• Re-check p99 latency and starvation counters after intervention.

Lessons learned

• Workload coverage beats clever microarchitecture

• Every change needs rollback triggers

TOPOLOGY COMPARISON SNAPSHOT
  candidate_A: 2D_mesh_6x6
  avg_hops: 3.1
  p99_latency_cycles: 74
  max_link_utilization: 0.83
  wirelength_norm: 1.00
  risk: directional_hotspot_under_dma_burst
  candidate_B: hierarchical_mesh
  avg_hops: 2.6
  p99_latency_cycles: 58
  max_link_utilization: 0.71
  wirelength_norm: 1.11
  risk: bridge_router_arb_complexity

Architecture deep dive

NoC is a queueing system — bandwidth, latency, and deadlock are coupled.

Concept diagram

NoC TOPOLOGY SKETCH

CPU0 ──┐      ┌── LLC0 ── DRAM0
       R0 ─── R1
CPU1 ──┘      │
              R2 ─── R3 ── GPU/DMA
              │      │
             NPU    LLC1 ── DRAM1

Look for: hot links, cyclic dependencies, VC starvation, and tail latency.

Metric graph

LATENCY DISTRIBUTION

p50    ██████  32 ns
p90    ████████████  71 ns
p99    ████████████████████████  210 ns
p99.9  █████████████████████████████████  480 ns

Averages hide QoS failures.

Metrics and artifacts

• link utilization

• average latency by master

• retry/backpressure counts

• QoS violation log

Mini case study

Average latency looks fine but tail latency spikes for CPU coherent reads when GPU DMA runs. QoS and separate VCs fix the starvation without doubling link width.

Debug branches

• If deadlock, check credit loops and routing restrictions first.

• If latency tail long, inspect arbitration and buffer depth.

Senior review question

Ask: what single metric would prove this concept is working or failing on your workload?

Key takeaways

• Connect every architecture claim to a workload and measurable metric.

• State verification and PPA impact before proposing design changes.

Common pitfalls

• Feature-driven design without MPKI/IPC/bandwidth evidence.

• Ignoring coherency and NoC traffic in cache and accelerator sizing.

Study notes

Re-read this topic with one concrete workload.