Accelerator Design Patterns — Extended Case Study

Extended case study

A review is called because a workload regresses after a Accelerator Design Patterns change.

Background

A stable baseline existed until a Accelerator Architectures change improved one benchmark and regressed a product workload on Accelerator pattern decision matrix.

Symptoms observed

• Regression in Accelerator pattern decision matrix

• Sim vs silicon disagreement

• Pressure to revert or ship risk

Investigation timeline

1. Freeze tags

2. Reproduce

3. Cluster

4. Experiment

5. Validate

6. Memo

Root cause

A hidden assumption in Accelerator Design Patterns failed under an unrepresented workload phase.

Fix and validation

• Bound change

• Replay workloads

• Check downstream impact

Lessons learned

• Workload coverage beats clever microarchitecture

• Every change needs rollback triggers

CASE STUDY — Accelerator Design Patterns
baseline/regressed/fixed metrics

Architecture deep dive

Accelerators win on locality and bandwidth contracts, not peak OPS alone.

Concept diagram

ACCELERATOR DATAFLOW

Host CPU ── commands ──► Queue / scheduler
   ▲                         │
   │ completion              ▼
Coherent memory ◄── DMA ── Local SRAM ──► Compute array
                         ▲       │
                         └ tiles ┘

Peak TOPS matters only when data reaches the array at the needed rate.

Metric graph

UTILIZATION BREAKDOWN

compute active   ██████████████████  58%
DMA wait         ██████████          31%
host sync        █████               15%
cache/coherency  ████                12%
idle bubbles     ███████             22%

Low utilization is usually a system integration problem.

Metrics and artifacts

• accelerator utilization

• DMA bandwidth

• kernel launch overhead

• coherency invalidation rate

Mini case study

NPU met TOPs target but end-to-end inference slow — DMA and weight fetch dominated. Architecture added on-chip SRAM tile and double-buffering.

Debug branches

• If util low, check launch overhead and host sync first.

• If BW high, examine weight layout and sparsity support.

Senior review question

Ask: what single metric would prove this concept is working or failing on your workload?

Key takeaways

• Connect every architecture claim to a workload and measurable metric.

• State verification and PPA impact before proposing design changes.

Common pitfalls

• Feature-driven design without MPKI/IPC/bandwidth evidence.

• Ignoring coherency and NoC traffic in cache and accelerator sizing.

Study notes

Re-read this topic with one concrete workload.