Golden Seed Sets

What is a golden seed set

A golden seed set is a curated subset of seeds that repeatedly catches critical failure classes with minimal runtime. It complements random sweeps by providing fast high-confidence gates on every commit.

[REG] seed tiers

  Tier 0 (sanity):  5-20 seeds, minutes, every PR
  Tier 1 (nightly): 100-500 seeds, overnight
  Tier 2 (deep):    thousands, milestone or weekend

  Golden set typically lives in Tier 0 and selected Tier 1

Selection criteria

• historical bug-catching rate

• coverage contribution diversity

• runtime cost per seed

• stability (low infra noise)

• protocol/feature distribution balance

Building and refreshing the set

Golden sets are not static forever. Refresh cadence (for example monthly) should rotate low-signal seeds out and promote high-signal newcomers.

# Example: produce candidate ranking from history
python3 tools/seed_rank.py \
  --history out/history/regression_runs.jsonl \
  --min-runs 30 \
  --output out/history/golden_seed_candidates.json

{
  "seed": 821734,
  "tests": ["axi_random_test", "coherency_stress_test"],
  "bug_signatures_caught": 4,
  "coverage_delta_percent": 0.27,
  "avg_runtime_sec": 112,
  "stability_score": 0.98
}

Promotion and demotion policy

1. Promote a seed if it catches at least one escaped bug class or yields persistent coverage gain.

2. Demote a seed if it provides no unique signal over 4+ refresh cycles.

3. Retain a minimum diversity quota per protocol family.

Operational use in CI

CI should run golden seeds first and fail fast on deterministic signatures. Full random sweeps remain necessary but can run post-merge or nightly.

# Pre-submit gate
python3 tools/run_regression.py \
  --test-list tests/pr_gate.txt \
  --seed-file seeds/golden_pr_gate.txt \
  --max-parallel 12

# Nightly expansion
python3 tools/run_regression.py \
  --test-list tests/nightly_full.txt \
  --seed-mode random \
  --seeds-per-test 200

[REG] [UVM] CI flow

  commit -> compile -> golden seeds -> pass? -> merge candidate
                                  |
                                  v
                             fail fast + replay cmd

  post-merge/nightly -> broad random seed sweep -> coverage merge -> trend update

Governance checklist

• Version-control golden seed files.

• Document source metrics for each seed in the set.

• Require owner approval for large set churn.

• Track gate runtime budget and enforce cap.

Key takeaways

• Golden seeds provide fast high-signal quality gates.

• Curate with measurable criteria: bug signal, coverage gain, runtime, stability.

• Refresh periodically; keep diversity and auditability.

• Use golden tiers with replay-ready logging for rapid CI triage.

Common pitfalls

• Treating golden seeds as permanent and never refreshing.

• Choosing seeds only by runtime and losing bug-catching power.

• Replacing random sweeps entirely with curated subsets.