Scenario: Intermittent Field-Like Failure - Interview Scenario

Scenario

A long-run workload intermittently triggers hangs after several hours, but directed short tests pass. Failures appear tied to concurrency and thermal drift rather than one deterministic command sequence.

OBSERVED METRIC
Failure incidence per soak-hour, recurrence under replay traffic, and time-to-isolation from first symptom to narrowed root-cause class.

45-MINUTE INTERVIEW FLOW
0-5: define setup context and KPI
5-15: classify first failing boundary
15-25: identify proving artifact packet
25-35: propose bounded fix with owner
35-45: define validation matrix and rollback

Common traps to avoid

• Labeling the issue as flaky infrastructure before preserving first-failure artifacts and environmental context.

• Running ad-hoc retries without hypothesis tagging, making logs non-comparable.

• Applying broad mitigations without owner-aligned rollback criteria and recurrence tracking.

Scenario debrief

Score responses on first-failure classification, evidence quality, owner assignment, and rollback-safe closure criteria.

symptom -> reproduce -> instrument -> isolate -> validate

reproducibility and recurrence trend

Debrief prompts

1. Which dependency boundary failed first and what artifact proves it?

2. Which root-cause class is most likely, and which class was explicitly disproven?

3. What bounded fix and validation matrix close residual risk?

Key takeaways

• Tie every bring-up claim to one reproducible setup state and one proving artifact.

• Prefer bounded fixes with clear owner and rollback trigger over broad multi-variable edits.

Common pitfalls

• Running parallel uncontrolled experiments and losing causality.

• Declaring closure without replaying across representative corners.

• Escalating severity before bench/setup hypotheses are disproven.