Replacement Policy and Miss Behavior — Interview Drills

[INT][ARCH][TOPIC]

Q: When is pseudo-LRU a poor choice for modern workloads?

A:
When set pressure is dominated by mixed streaming and bursty reuse, where adaptive insertion can reduce dead-on-arrival fills better than age approximation alone.

FOLLOW-UP TRAP: Assuming pseudo-LRU is always near-optimal.

[INT][ARCH][TOPIC]

Q: How do you detect prefetch pollution in MPKI regressions?

A:
Compare demand-hit impact with prefetch eviction contribution and dead-on-fill ratio from reuse-distance analysis.

FOLLOW-UP TRAP: Looking only at total misses.

[INT][ARCH][TOPIC]

Q: What makes replacement policy change risky in production?

A:
It can shift fairness and QoS under diverse tenants; rollout needs telemetry, staged deployment, and safe fallback.

FOLLOW-UP TRAP: Deploying globally from lab-only validation.

[INT][ARCH][STAFF]

Q: You inherit a late-stage Replacement Policy and Miss Behavior failure one week before release. What do you do in the first hour?

A:
Freeze the workload/model/RTL tag, name the failing metric (LLC miss decomposition + reuse distance report), confirm counter setup, cluster the issue by structure or workload phase, assign the first experiment, and publish a validation/owner plan before changing architecture.

FOLLOW-UP TRAP: Jumping directly to a larger cache, wider pipe, or extra NoC link without preserving evidence.

[INT][ARCH][STAFF]

Q: When would you stop trying to improve Replacement Policy and Miss Behavior and escalate?

A:
Escalate when the remaining risk crosses ownership boundaries, consumes shared margin, changes signed-off assumptions, or threatens SoC QoS enforcement, firmware scheduling, and customer-perceived performance stability rely on miss behavior.. Bring exact report lines and options, not vague concern.

FOLLOW-UP TRAP: Escalating without data or continuing alone after a cross-team decision is needed.

VISUAL MODEL — Memory Hierarchy / Replacement Policy and Miss Behavior

        workload / trace
              │
              ▼
   metric symptom (LLC miss decomposition + reuse distance report)
              │
              ▼
     likely microarchitectural mechanism
              │
      ┌───────┼────────┐
      ▼       ▼        ▼
  pipeline  memory    fabric/coherency
  stalls    misses    queues / ordering
      │       │        │
      └───────┼────────┘
              ▼
        bounded design change
              │
              ▼
   validation workload + PPA regression

MEMORY HIERARCHY

Core
 ├─ L1I / L1D  (cycles: 1-4, tiny, latency critical)
 ├─ L2         (cycles: 8-20, private or cluster)
 ├─ LLC / SLC  (shared, bandwidth + coherency point)
 ├─ NoC        (queueing + arbitration)
 └─ DRAM/HBM   (large penalty, high energy)

AMAT = hit_time + miss_rate × miss_penalty
But senior analysis also asks: MLP, bandwidth, QoS, and tail latency.

MISS PENALTY WATERFALL

L1 hit          ██  3 cyc
L2 hit          ████████  12 cyc
LLC hit         ███████████████  32 cyc
DRAM miss       ████████████████████████████████████  180 cyc

Small MPKI can still dominate if miss penalty is huge.