DenseChip

Computer Architecture · All levels

Scenario: Accelerator Starvation

An NPU hits peak TOPs in isolation but end-to-end inference is 40% slower than target. Marketing wants to announce the peak number anyway.

Timed scenario (45 minutes)

An NPU hits peak TOPs in isolation but end-to-end inference is 40% slower than target. Marketing wants to announce the peak number anyway.

Timer breakdown

  1. 0–5 min: define end-to-end metric and workload

  2. 5–15 min: util%, DMA BW, launch overhead, weight fetch

  3. 15–25 min: locality, tiling, double-buffering hypotheses

  4. 25–35 min: SRAM tile / DMA / software contract options

  5. 35–45 min: what metric is safe to publish vs what is misleading

diagram
RUBRIC (score 1–5 each)

  Metric clarity — names IPC, latency, bandwidth, or stall source
  Mechanism depth — explains pipeline, cache, NoC, or coherency cause
  Tradeoff quality — states PPA and verification impact
  Validation mindset — names workload and regression checks

STRONG HIRE: METRIC → mechanism → experiment without prompting
NO HIRE: buzzwords without measurable impact

Interviewer follow-up traps

  • Quote peak TOPs only

  • Ignore host sync

  • Blame compiler without data

Strong hire signals

  • Names workload and baseline metric first

  • Proposes cheapest experiment before design change

  • States PPA and verification regression impact

Scenario debrief

Mark where you named metric, mechanism, experiment, and validation.

diagram
WHITEBOARD ANSWER SHAPE

1. Workload and metric
2. Block diagram
3. Bottleneck hypothesis tree
4. Experiment plan
5. Design options and tradeoffs
6. Validation / regression

If you skip step 1, the rest sounds generic.
diagram
ANSWER QUALITY LADDER

5/5  metric + mechanism + experiment + tradeoff + validation
4/5  correct mechanism, weak validation
3/5  concept knowledge, no workload
2/5  feature list
1/5  buzzwords

Debrief prompts

  1. What metric did you open with?

  2. Which experiment was cheapest?

  3. What PPA tradeoff did you state?

Key takeaways

  • Connect every architecture claim to a workload and measurable metric.

  • State verification and PPA impact before proposing design changes.

Common pitfalls

  • Feature-driven design without MPKI/IPC/bandwidth evidence.

  • Ignoring coherency and NoC traffic in cache and accelerator sizing.

Practice this lesson