Part 6 · Agents & Protocol IP · Intermediate

VIP Test Suite: Qualification Matrix for Reuse Confidence

Building and maintaining a VIP qualification suite that validates protocol correctness, mode behavior, configuration interactions, and compatibility across tools.

What a qualification suite must prove

A reusable VIP suite must prove more than protocol legality in one simulator. It should cover feature correctness, active/passive behavior, error handling, and compatibility promises.

Think of qualification as an executable contract for every release candidate.

diagram

[VIP][CI] qualification dimensions

protocol:
  legal traffic, illegal traffic detection, corner timing

mode:
  active drive correctness
  passive observation correctness

configuration:
  defaults, boundaries, invalid input handling

compatibility:
  simulator versions, UVM versions, parameter variants

diagram

[AGT] suite composition

smoke tier:
  fast sanity for PR gating

feature tier:
  each protocol feature with directed checks

stress tier:
  constrained-random and long-run stability

compat tier:
  simulator/UVM matrix jobs

Qualification should align with published VIP guarantees.
Cover both correctness and misuse detection paths.
Split tests by tier for fast feedback and deep confidence.

Core test categories and examples

Define explicit categories so coverage gaps are visible. Map each public capability to one or more tests.

diagram

[VIP] core category checklist

1) reset behavior
2) single transfer read/write
3) burst/streaming behavior
4) response/error signaling
5) backpressure and wait states
6) active/passive parity where applicable
7) cfg boundary values
8) invalid cfg rejection
9) monitor decode integrity

diagram

[UVM] active/passive parity case

test A (active):
  sequence drives N legal transfers
  monitor should publish N matching items

test B (passive):
  external BFM drives same transfers
  monitor should publish equivalent items

parity criterion:
  decoded transaction semantics align

systemverilog

class vip_cfg_invalid_test extends uvm_test;
  `uvm_component_utils(vip_cfg_invalid_test)
  function void build_phase(uvm_phase phase);
    vip_cfg cfg = vip_cfg::type_id::create("cfg");
    cfg.timeout_cycles = 0; // illegal
    uvm_config_db#(vip_cfg)::set(this, "env.vip_agt", "cfg", cfg);
    super.build_phase(phase);
  endfunction
endclass

Map each documented capability to dedicated tests.
Include negative tests for config and protocol misuse.
Compare active/passive decode semantics where feasible.

Compatibility matrix strategy

Consumers often run different simulators and UVM flavors. A declared support matrix requires regular CI execution and transparent reporting.

diagram

[CI][VIP] matrix dimensions

simulators:
  vcs, xrun, questa (as supported)

UVM versions:
  UVM-1.2, IEEE-2020 (as supported)

parameter variants:
  narrow and wide data paths
  single and multi-channel configs

diagram

[CI] matrix policy

PR gates:
  smoke on primary simulator

nightly:
  feature tier on full matrix

pre-release:
  stress + full matrix + compatibility report publication

diagram

[DOC] matrix report snippet

job                         result   duration
------------------------------------------------
vcs_uvm12_smoke             PASS     4m
xrun_ieee2020_feature       PASS     18m
questa_ieee2020_smoke       PASS     6m
vcs_uvm12_stress            PASS     52m

State support matrix explicitly and test it continuously.
Use tiered schedules to balance runtime and coverage.
Publish matrix results with releases for consumer confidence.

Suite maintainability and signal-to-noise

Qualification suites must remain stable and informative. Flaky tests erode trust; opaque failures slow adoption.

diagram

[VIP][CI] maintainability practices

- deterministic seeds for smoke tests
- bounded random for feature tests
- dedicated long-run stress seeds tracked separately
- clear pass/fail criteria per test
- stable, searchable failure tags

diagram

[DEBUG] failure triage metadata

include in logs:
  vip version
  test name and seed
  simulator and UVM version
  mode (active/passive)
  parameter set (ADDR_W/DATA_W)
  cfg summary

diagram

[RELEASE] qualification sign-off questions

1) did every required category execute?
2) did matrix jobs pass within support policy?
3) any known flaky tests unresolved?
4) are failures fully triaged with owner/action?
5) is report attached to release artifact?

Key takeaways

VIP qualification is executable evidence for reuse claims.
Tiered suites balance developer velocity and release confidence.
Compatibility matrices must be tested, not merely documented.
Maintainability practices keep qualification signals trustworthy.

Common pitfalls

Treating one-simulator smoke as sufficient qualification.
Skipping negative tests for cfg and protocol misuse.
Allowing flaky tests to accumulate without ownership.
Publishing release tags without attached qualification reports.

Practice this lessonQuestions tagged for this topic in the bank