Accelerator Design Patterns — Theory Deep Dive

Foundational theory

Accelerator Design Patterns sits inside Accelerator Architectures and changes how workload pressure becomes stalls, bandwidth, latency, and power. Pattern selection balances operation regularity, control divergence, precision behavior, and expected software runtime model.

Core concepts explained

• Map workload semantics to reusable accelerator patterns such as systolic, SIMD, reduction, and streaming pipelines.

• Primary evidence: Accelerator pattern decision matrix

• Downstream: Compiler/runtime architecture and verification scope.

• Risk: Wrong pattern lock-in creates multi-quarter schedule slips and low silicon ROI.

• Classify kernels by data reuse, dependency depth, and synchronization style.

• Estimate pattern fit using utilization under realistic shape and sparsity distributions.

• Include software stack complexity and compiler maturity in feasibility scoring.

Why this matters in real chips

In production programs, Accelerator Design Patterns appears when workloads miss IPC, latency, or power targets. Mechanism-first reasoning prevents expensive architecture churn.

Mental model

THEORY STACK — Accelerator Design Patterns
Workload -> mechanism -> metric (Accelerator pattern decision matrix) -> bounded decision

Worked intuition

1. Name the workload class.

2. Name the metric that moves first.

3. Identify the responsible structure.

4. Check software/coherency amplification.

5. Propose the smallest reversible experiment.

Common misconceptions

• Using average metrics when tails dominate.

• Tuning one benchmark without product workload mix.

• Ignoring verification and software cost.

• Selecting patterns from peak TOPS comparisons only.

• Ignoring control divergence penalties in real models.

Key takeaways

• Explain Accelerator Design Patterns with mechanism and metric.

Architecture deep dive

Accelerators win on locality and bandwidth contracts, not peak OPS alone.

Concept diagram

ACCELERATOR DATAFLOW

Host CPU ── commands ──► Queue / scheduler
   ▲                         │
   │ completion              ▼
Coherent memory ◄── DMA ── Local SRAM ──► Compute array
                         ▲       │
                         └ tiles ┘

Peak TOPS matters only when data reaches the array at the needed rate.

Metric graph

UTILIZATION BREAKDOWN

compute active   ██████████████████  58%
DMA wait         ██████████          31%
host sync        █████               15%
cache/coherency  ████                12%
idle bubbles     ███████             22%

Low utilization is usually a system integration problem.

Metrics and artifacts

• accelerator utilization

• DMA bandwidth

• kernel launch overhead

• coherency invalidation rate

Mini case study

NPU met TOPs target but end-to-end inference slow — DMA and weight fetch dominated. Architecture added on-chip SRAM tile and double-buffering.

Debug branches

• If util low, check launch overhead and host sync first.

• If BW high, examine weight layout and sparsity support.

Senior review question

Ask: what single metric would prove this concept is working or failing on your workload?

Key takeaways

• Connect every architecture claim to a workload and measurable metric.

• State verification and PPA impact before proposing design changes.

Common pitfalls

• Feature-driven design without MPKI/IPC/bandwidth evidence.

• Ignoring coherency and NoC traffic in cache and accelerator sizing.

Study notes

Re-read this topic with one concrete workload.