Cache & Memory Hierarchy

Section goal

Cache levels, translation structures, and prefetch engines that shape effective memory latency and bandwidth.

How to study this section

1. Start with each topic hub to establish mechanism-first framing.

2. Use reports and debug pages to separate symptoms from root cause.

3. Practice worked examples and interview drills under fixed metadata.

4. Close with checklist and silicon impact before signoff claims.

Topics

1. l1-instruction-data-caches/ - L1 Instruction/Data Caches

2. l2-l3-hierarchy-design/ - L2/L3 Hierarchy Design

3. tlb-and-address-translation/ - TLB and Address Translation

4. prefetch-and-stream-buffers/ - Prefetch and Stream Buffers

Related topics

• CPU Design course hub

CPU deep dive

Memory hierarchy closure needs cache, TLB, and prefetch policy to be tuned together for real latency tails.

Concept diagram

MEMORY + TRANSLATION STACK

L1I/L1D -> L2 -> LLC -> DRAM
   |       |      |
 ITLB/DTLB hierarchy + page walkers

Metric graph

LATENCY TAIL CONTRIBUTORS

cache miss chains      █████
translation misses     ████
coherence interference ███

Reports and artifacts

• L1/L2/LLC latency stack

• TLB walk profile

• prefetch usefulness report

• memory tail percentile dashboard

Mini case study

Prefetch aggressiveness improved average misses but worsened p99 latency by polluting LLC and stressing page walkers.

Debug branches

• Tag misses by source: capacity, conflict, translation, or coherence

• Track TLB shootdowns and page-size behavior with workload phases

• Evaluate prefetch policy on tail latency, not just average CPI

Senior review question

Ask: which CPI/latency evidence proves this topic is truly closed beyond synthetic benchmarks?