Dataflow Architecture Choices — Extended Case Study

Extended case study

A review is called because a workload regresses after a Dataflow Architecture Choices change.

Background

A stable baseline existed until a Accelerator Architectures change improved one benchmark and regressed a product workload on architecture KPI dashboard.

Symptoms observed

• Regression in architecture KPI dashboard

• Sim vs silicon disagreement

• Pressure to revert or ship risk

Investigation timeline

1. Freeze tags

2. Reproduce

3. Cluster

4. Experiment

5. Validate

6. Memo

Root cause

Hybrid mode selection by runtime shape recovers tail efficiency while preserving peak throughput.

Fix and validation

• Replay workloads with phase markers and dataflow telemetry enabled.

• Compare compute-idle windows against transfer/synchronization events.

• Identify tiles or stages with repeated spill/refill churn.

• Test alternative scheduling or tile geometry under identical constraints.

• Lock a mode-selection policy with measurable guardrails.

Lessons learned

• Workload coverage beats clever microarchitecture

• Every change needs rollback triggers

CASE STUDY — Dataflow Architecture Choices
baseline/regressed/fixed metrics

Architecture deep dive

Accelerators win on locality and bandwidth contracts, not peak OPS alone.

Concept diagram

ACCELERATOR DATAFLOW

Host CPU ── commands ──► Queue / scheduler
   ▲                         │
   │ completion              ▼
Coherent memory ◄── DMA ── Local SRAM ──► Compute array
                         ▲       │
                         └ tiles ┘

Peak TOPS matters only when data reaches the array at the needed rate.

Metric graph

UTILIZATION BREAKDOWN

compute active   ██████████████████  58%
DMA wait         ██████████          31%
host sync        █████               15%
cache/coherency  ████                12%
idle bubbles     ███████             22%

Low utilization is usually a system integration problem.

Metrics and artifacts

• accelerator utilization

• DMA bandwidth

• kernel launch overhead

• coherency invalidation rate

Mini case study

NPU met TOPs target but end-to-end inference slow — DMA and weight fetch dominated. Architecture added on-chip SRAM tile and double-buffering.

Debug branches

• If util low, check launch overhead and host sync first.

• If BW high, examine weight layout and sparsity support.

Senior review question

Ask: what single metric would prove this concept is working or failing on your workload?

Key takeaways

• Connect every architecture claim to a workload and measurable metric.

• State verification and PPA impact before proposing design changes.

Common pitfalls

• Feature-driven design without MPKI/IPC/bandwidth evidence.

• Ignoring coherency and NoC traffic in cache and accelerator sizing.

Study notes

Re-read this topic with one concrete workload.