Regression Seeds & Reproducibility

Overview

Randomization is useful only when failures are reproducible . A nightly run might execute thousands of tests over hundreds of seeds. If one fails but the exact seed is not captured, triage stalls. If the seed is captured but replay is inconsistent across tools, debug quality drops. This hub defines an end-to-end seed discipline that makes every failure replayable and every seed's contribution measurable.

The five lessons in this topic walk from seed injection at simulator command line through persistent logging, replay automation, intermittent failure forensics, and coverage-aware seed selection. The goal is practical: one failure should become one deterministic local replay command.

Lessons in this topic

1. Simulator Seed Control - normalize tool-specific seed flags and test-level seed policy.

2. Logging & Seed Replay - record seed/test/build metadata and recreate failures exactly.

3. Intermittent Failure Workflow - move from flaky symptom to deterministic root cause.

4. Seeds + Coverage Merge - understand how per-seed UCDB artifacts combine into closure.

5. Golden Seed Sets - maintain small stable seed suites for pre-submit and smoke gates.

Seed lifecycle map

[REG] [UVM] regression seed lifecycle

  Plan matrix      Launch sim       Runtime log       Failure bucket
  -----------      ----------       -----------       -------------
  test x seed ---> +seed option ---> emit seed id ---> signature hash
       |                  |               |                  |
       |                  |               |                  v
       |                  |               |             replay queue
       |                  |               v
       |                  |        artifact manifest
       |                  v
       |           simulator RNG stream
       |
       v
  coverage merge <--- per-seed UCDB

  [CHECK] Always persist: test, seed, simulator version, git sha, plusargs

Treat seed metadata as first-class debug data , not as console noise. A mature flow stores seed rows in the same place as pass/fail and coverage metrics.

# Example matrix launcher shape
python3 tools/run_regression.py \
  --test-list tests/nightly_smoke.txt \
  --seed-mode random \
  --seeds-per-test 20 \
  --emit-manifest out/regression_manifest.json

Key takeaways

• Every random failure must be replayable with one deterministic command.

• Capture seed metadata at launch and at runtime report level.

• Use seed-aware triage and seed-aware coverage merge, not pass/fail only.

• Keep a curated golden seed subset for rapid quality gates.

Common pitfalls

• Using random seeds without persistent logging.

• Assuming the simulator default seed is stable across tool versions.

• Treating flaky failures as noise instead of replaying and classifying by signature.