Computer Architecture · All levels

Dataflow Architecture Choices — Worked Example

Worked Example for Dataflow Architecture Choices (Accelerator Architectures).

Scenario

Output-stationary wins on benchmark kernels but collapses on small-batch online inference due to refill overhead.

Timeline

Metric fails at review meeting
Engineer captures metric report, trace snippet, and workload phase
Root cause traced to incorrect assumption from prior stage
Minimal architecture change or policy experiment applied and documented
Workload regression matrix re-run on the tagged model

Outcome

Hybrid mode selection by runtime shape recovers tail efficiency while preserving peak throughput.

Senior debrief

After solving the example, write the debrief a lead would expect: what changed, why it worked, what could regress, and what permanent methodology update prevents recurrence.

diagram

STAFF ARCHITECTURE REVIEW MEMO — Accelerator Architectures / Dataflow Architecture Choices

1. Current state
   - Failing / watched metric: Accelerator Architectures closure dashboard
   - Workload / benchmark / trace: <fill before review>
   - Model tag, RTL tag, simulator version, PMU setup: <fill before review>
   - Scope: core, cache level, NoC path, coherency domain, accelerator, or SoC budget

2. Root-cause hypothesis
   - Most likely mechanism: <name pipeline/cache/NoC/coherency/perf mechanism>
   - Competing hypothesis: <name the second plausible cause>
   - Evidence still missing: <counter, trace, waveform, model sweep, or workload slice>

3. Proposed action
   - Minimal reversible change: <microarchitecture, policy, sizing, traffic, or software contract change>
   - Expected improvement: <metric delta>
   - Regression risk: late-stage schedule slip, silicon risk, or cross-stage regression

4. Regression and signoff
   - Re-run: Accelerator Architectures closure dashboard
   - Must not regress: Memory subsystem sizing and runtime policy complexity.
   - Decision owner: architecture owner

Before / after metric graph

diagram

METRIC TREND GRAPH — Dataflow Architecture Choices

IPC / throughput
  ^
  |                  target
  |                 ─ ─ ─ ─ ─ ─ ─
  |            ● after bounded fix
  |         /
  |    ● baseline
  |  /
  |● failing run
  +---------------------------------> experiment index
    bad tag       hypothesis        accepted fix

Readout rule:
  - one dot is not a conclusion
  - compare against same workload, seed, model tag, and counter setup
  - explain why the fix moved the metric, not just that it moved

Architecture deep dive

Accelerators win on locality and bandwidth contracts, not peak OPS alone.

Concept diagram

diagram

ACCELERATOR DATAFLOW

Host CPU ── commands ──► Queue / scheduler
   ▲                         │
   │ completion              ▼
Coherent memory ◄── DMA ── Local SRAM ──► Compute array
                         ▲       │
                         └ tiles ┘

Peak TOPS matters only when data reaches the array at the needed rate.

Metric graph

diagram

UTILIZATION BREAKDOWN

compute active   ██████████████████  58%
DMA wait         ██████████          31%
host sync        █████               15%
cache/coherency  ████                12%
idle bubbles     ███████             22%

Low utilization is usually a system integration problem.

Metrics and artifacts

accelerator utilization
DMA bandwidth
kernel launch overhead
coherency invalidation rate

Mini case study

NPU met TOPs target but end-to-end inference slow — DMA and weight fetch dominated. Architecture added on-chip SRAM tile and double-buffering.

Debug branches

If util low, check launch overhead and host sync first.
If BW high, examine weight layout and sparsity support.

Senior review question

Ask: what single metric would prove this concept is working or failing on your workload?

Key takeaways

Connect every architecture claim to a workload and measurable metric.
State verification and PPA impact before proposing design changes.

Common pitfalls

Feature-driven design without MPKI/IPC/bandwidth evidence.
Ignoring coherency and NoC traffic in cache and accelerator sizing.

Study notes

Re-read this topic with one concrete workload.

Practice this lesson