Step 6: Tests, Regression & Closure

The test list

One environment, many tests — each test is a thin recipe that picks scenarios, knobs, and length. The variants map straight back to plan rows.

TEST VARIANTS

  Test            │ Recipe                              │ Plan rows
  ────────────────┼─────────────────────────────────────┼───────────────
  smoke           │ 20 pushes, 20 pops, fixed seed      │ F-01,04 (sanity)
  stress_mixed    │ 2000 txns SCN_MIXED, max_gap 0-1    │ F-01,07, wrap
  burst_drain     │ fill→full, hold, drain→empty, x10   │ F-02,03,05,06,10
  corner_simul    │ park at full / at empty, OP_BOTH    │ F-07,08
  reset_storm     │ random resets mid-traffic, restart  │ F-09
  error_probe     │ in_valid at full, out_ready at      │ F-05,06
                  │ empty held for windows              │
  param_min       │ rebuild DEPTH=2, run all the above  │ F-11
  param_w1        │ rebuild WIDTH=1, smoke + stress     │ F-11

// a test is a recipe — example: burst_drain
task test_burst_drain(generator #(8) gen, fifo_sb #(8) sb);
  repeat (10) begin
    gen.run_scenario(SCN_BURST_FILL, 24);    // > DEPTH: park at full
    gen.run_scenario(SCN_BACKPRESSURE, 10);  // hold full under pressure
    gen.run_scenario(SCN_DRAIN, 24);         // > DEPTH: park at empty
  end
endtask

Seed sweep and regression plan

Each random test runs with many seeds — different seeds walk different boundary timings and gap patterns. The sweep is tiered so checkins get fast feedback while nights buy coverage.

REGRESSION TIERS

  per-checkin   smoke + 1 seed of stress_mixed            ~2 min
  nightly       all tests × 10 seeds × {DEPTH=16}         ~1 hr
                + param_min, param_w1 × 3 seeds
  weekly soak   stress_mixed × 200 seeds, reset_storm     coverage
                × 50 seeds, merge ALL coverage            closure run

  bookkeeping per run: seed, test, RTL version, pass/fail, ucdb path
  → failures reproducible with one command; coverage merge automatic

The hole analysis pass

After the soak merge, every uncovered bin gets a written disposition : a directed fix, a constraint change, or a justified waiver. This capstone's merge produced three findings — typical of what hole analysis looks like in practice.

1. x_op_state.both_at_full at 0 hits — the generator never offered push and pop in the same cycle while parked at full. Fix: corner_simul test parks at full then issues OP_BOTH bursts. Re-run: covered, and it exposed a real RTL bug (push accepted at full when pop fired — the spec Q-003 case, implemented wrong).

2. cp_count.almost_full hit only 12 times in 200 seeds — legal but thin; mixed traffic crosses the boundary too fast. Fix: SCN_BACKPRESSURE weight raised in stress_mixed; now hundreds of hits.

3. param_w1 × reset_storm uncovered nothing new after 3 seeds — waiver: WIDTH=1 has no width-dependent reset logic; documented, approved, closed.

Sign-off summary and the interview retrospective

SIGN-OFF SUMMARY — fifo v1.2

  plan rows           11/11 closed (10 covered+checked, 1 waived: F-11/W1 note)
  functional coverage 100% of goal after merge (47 bins, 2 waived w/ rationale)
  assertions          9 properties, 0 failures final regression,
                      0 vacuous-only passes (antecedent covers all hit)
  scoreboard          1.2M words compared across soak, 0 mismatches
  bugs found          3 RTL (incl. push-at-full-with-pop), 2 TB, all closed
  bug-rate            0 new bugs in final 2 weekly soaks
  residual risk       async use unverified (out of scope — sync FIFO only)

What this capstone proves in an interview

• You can run the full lifecycle solo: spec → plan → environment → closure → sign-off argument, with artifacts for each stage.

• You separate concerns: stimulus vs checking vs assertions vs coverage, each with a distinct job and failure signature.

• You design for corners deliberately — the boundary bins, the simultaneity cross, the reset storm — rather than hoping randomness arrives.

• Your best story: the hole-analysis finding where a coverage gap (both_at_full) led to a real RTL bug — coverage-driven verification working exactly as advertised.

• Every senior scenario question — "verify a FIFO", "how do you know you're done", "tell me about a bug you found" — now has a concrete, defensible answer from this project.

Key takeaways

• Tests are thin recipes over one environment — variants map one-to-one to plan rows.

• Tiered regression: fast checkin, broad nightly, coverage-closing weekly soak.

• Every hole gets a disposition: directed fix, constraint change, or documented waiver.

• Sign-off is an evidence bundle: plan closure, merged coverage, assertion health, scoreboard volume, converged bug rate.

• The capstone is your interview portfolio — narrate it stage by stage.

Common pitfalls

• Declaring closure from a single seed's coverage — merge the soak first.

• Waiving holes without written rationale — indistinguishable from negligence at review.

• Skipping the param_min config — DEPTH=2 found pointer bugs DEPTH=16 never showed.

• Treating the capstone as code only — the plan and sign-off artifacts are half its value.