Extended case study

Integration review: spec feature coverage, compliance pass rate, waiver count regresses after a change touching Compliance & Coverage Plan.

Background

Baseline traffic passed compliance and performance targets. A bridge update, firmware change, or clock/reset tweak introduced intermittent failures visible only under mixed traffic.

Symptoms observed

• Regression in spec feature coverage, compliance pass rate, waiver count

• VIP warning followed by software timeout (symptom lag)

• Directed tests pass; stress or product replay fails

• Two teams disagree because they look at different layers

Investigation timeline

1. Hour 0: freeze sim tag, firmware, and spec revision

2. Hour 1: capture first failing transaction with ID/address

3. Hour 2: correlate waveform, VIP monitor, and counter

4. Hour 3: classify: rule violation vs config vs timing vs load

5. Hour 4: reduce to 3-transaction minimal sequence

6. Hour 5: bounded RTL or register fix + regression list

7. Hour 6: compliance replay + product workload signoff memo

Root cause

The failing behavior traced to a violated assumption in Compliance & Coverage Plan: compliance maps spec requirements to tests, assertions, coverage bins, exclusions, and signed waivers.

Fix and validation

• Minimal reversible change at the owning boundary

• Re-run requirement trace matrix, coverage dashboard, waiver log on failing and baseline seeds

• Compliance suite + mixed-traffic regression

• Document software-visible impact and waiver if any

Lessons learned

• First bad transaction beats loudest timeout

• Layer alignment across RTL, VIP, firmware, and analyzer

• Performance and correctness regressions need separate evidence

CASE STUDY METRICS — Compliance & Coverage Plan

baseline     spec feature coverage, compliance pass rate, waiver count: within target
regressed    spec feature coverage, compliance pass rate, waiver count: fails product threshold
after fix    spec feature coverage, compliance pass rate, waiver count: restored + compliance PASS
residual risk: document waiver or monitor in field

Sequence under stress

SEQUENCE — Compliance & Coverage Plan

  initiator            interconnect/PHY            target
      |  request (id) ------->  |                     |
      |                         |  forward ----------> |
      |                         |                     | work
      |                         |  <---- response ---- |
      |  <----- complete ------ |                     |
      |
   metric captured here: spec feature coverage, compliance pass rate, waiver count

Protocol deep dive

Verification closes the gap between 'works in directed test' and 'legal under all stressed traffic'.

Concept diagram

VERIFICATION CLOSURE LOOP

spec clause -> test -> assertion -> coverage -> waiver -> signoff
                  ^                           |
                  +--------- gap found --------+

Metric graph

COVERAGE vs ESCAPE RATE

escapes
  |*
  | *
  |  *
  |   **  <- knee: more random helps
  |     ****
  +----------------> constrained-random depth

Metrics and artifacts to collect

• compliance pass rate

• coverage closure

• scoreboard mismatch rate

• assertion fire count

• waiver log

Mini case study

Block passed VIP compliance but chip failed: system test omitted cross-master ID reuse through a bridge. Scoreboard key did not include upstream port ID.

Debug branches

• If VIP pass but silicon fail, check integration assumptions.

• If scoreboard noisy, fix key (ID+port+addr).

• If coverage plateau, add compliance gap analysis not more repeats.

Senior review question

Ask: what is the first transaction that deviates, and which spec rule does it test?

Key takeaways

• Connect every protocol claim to a transaction identity and measurable metric.

• Store the artifact (waveform, log, counter) next to every signoff decision.

Common pitfalls

• Debugging timeouts without finding the first bad transaction.

• Quoting peak bus width without payload efficiency and retry overhead.

• Treating VIP compliance as a substitute for system integration replay.

Field case notes

Mixed traffic exposed a bug that single-master directed tests missed for three weeks.