Cache Organization and Access Path — Review Checklist

Review gate

1. Hit rate and hit latency are both reported per cache level.

2. Miss taxonomy and workload mix are documented in review.

3. Perf/watt impact is measured for at least two operating points.

4. Coherence and implementation complexity are included in acceptance criteria.

5. Decision is segmented by SKU if workload behavior diverges.

Smoke check (5 minutes)

• Every checklist item has an owner

• Failed items have owner, mitigation, and decision record

Definition of done for a senior owner

1. The exact workload, model/RTL tag, counter setup, and analysis window are recorded.

2. The primary metric is clean, improved, or accepted as a documented product tradeoff: Cache hierarchy KPI dashboard.

3. The change is explained by mechanism, not by architecture folklore.

4. Regression coverage includes the obvious downstream domains: Pipeline stall behavior, NoC traffic profile, and SoC thermal budget depend on cache organization..

5. Residual risk has an owner, approval path, and expiration date.

6. The lesson is captured as a methodology guardrail if it can recur.

Smoke check (5 minutes)

• Could another engineer reproduce the conclusion from the notes alone?

• Would you sign this off if the design came from another team?

Review visual

TRADEOFF MATRIX — Cache Organization and Access Path

+----------------------+----------------------+----------------------+----------------------+
| Option               | Helps                | Can hurt             | Validation needed    |
+----------------------+----------------------+----------------------+----------------------+
| Larger / wider block | peak perf, miss rate | area, power, timing  | workload sweep       |
| Smarter policy       | hit rate, QoS, IPC   | verification risk    | corner cases + PMU   |
| More buffering       | latency tails, stalls| deadlock, leakage    | stress traffic tests |
| Software contract    | locality, ordering   | portability, APIs    | production workload  |
+----------------------+----------------------+----------------------+----------------------+

Senior rule: pick the smallest change that proves or disproves the mechanism.

Architecture deep dive

Cache hierarchy trades area and power for AMAT and bandwidth.

Concept diagram

MEMORY HIERARCHY

Core
 ├─ L1I / L1D  (cycles: 1-4, tiny, latency critical)
 ├─ L2         (cycles: 8-20, private or cluster)
 ├─ LLC / SLC  (shared, bandwidth + coherency point)
 ├─ NoC        (queueing + arbitration)
 └─ DRAM/HBM   (large penalty, high energy)

AMAT = hit_time + miss_rate × miss_penalty
But senior analysis also asks: MLP, bandwidth, QoS, and tail latency.

Metric graph

MISS PENALTY WATERFALL

L1 hit          ██  3 cyc
L2 hit          ████████  12 cyc
LLC hit         ███████████████  32 cyc
DRAM miss       ████████████████████████████████████  180 cyc

Small MPKI can still dominate if miss penalty is huge.

Metrics and artifacts

• MPKI per level

• L2/L3 bandwidth utilization

• replacement policy stats

• prefetch accuracy

Mini case study

Doubling L2 size reduces capacity misses but IPC improves only 3% because conflict misses dominate a shared workload. Fix data layout and false sharing before more SRAM.

Debug branches

• If MPKI high but bandwidth low, footprint may exceed capacity.

• If bandwidth saturated, coherency or DMA may be the real limit.

Senior review question

Ask: what single metric would prove this concept is working or failing on your workload?

Key takeaways

• Connect every architecture claim to a workload and measurable metric.

• State verification and PPA impact before proposing design changes.

Common pitfalls

• Feature-driven design without MPKI/IPC/bandwidth evidence.

• Ignoring coherency and NoC traffic in cache and accelerator sizing.

Study notes

Re-read this topic with one concrete workload.