A year into a project, the DMA bench has 140 tests. Forty of them are copies of dma_rand_test with one constraint changed. Two of them are, by accident, the same test under different names. The subsystem team wrote their own program_dma because the block helpers would not work for them. Nobody planned any of this; each test was a reasonable addition on the day it was written. What decisions, made early, would have kept the library small and the regression honest?

The idea: a test library is a product with users

A test library is the set of test classes plus the lists that say which of them run, with how many seeds, and when. Its users are the people who add tests, the people who read regression results, and the people one level up who want to reuse your stimulus. Each of them needs something different: an obvious place to add a scenario, a name that says what failed, and stimulus that is not locked inside a test class.

A good kitchen's recipe book is the analogy. Base preparations (stocks, doughs) are written once and referenced. Dishes are named for what they are, so a cook can find one. Variations are written as "as above, but", not copied out in full. And the menu, what is served tonight, is a separate list from the book. Tests are the dishes, sequences are the base preparations, and regression lists are the menus.

The picture: a library that grows without copying

Build it: five decisions

1. Depth: one base, plus a base per family

Keep the hierarchy shallow: the project base test, then an intermediate base only where three or more tests share setup or checking, then leaf tests. Error tests are the classic family: they all program one length and send fewer words, they all expect STATUS.ERR instead of DONE, and they all check that the error was reported. That shared expectation belongs in one class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

// snippet
// A family base: every error test runs one job and expects ERR, not DONE.
class dma_err_base_test extends dma_base_test;
  `uvm_component_utils(dma_err_base_test)
  int unsigned prog_len = 8;     // what the test programs
  int unsigned sent_len = 3;     // what it actually sends

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  task run_phase(uvm_phase phase);
    dma_job_cfg job = dma_job_cfg::type_id::create("job");
    bit [31:0] status;
    phase.raise_objection(this);
    if (!job.randomize() with { len == local::prog_len; })
      `uvm_fatal("RAND", "dma_job_cfg randomize failed")
    run_job(job, sent_len, status);
    if (status[2] !== 1'b1) `uvm_error("NOERR", $sformatf("%s: ERR not set, STATUS=%08h", get_type_name(), status))
    #100ns;
    phase.drop_objection(this);
  endtask
endclass

A leaf error test now sets two numbers, or registers one override, and inherits the expectation. Deeper hierarchies (a base for every feature, a base for every base) make it hard to know which run_phase actually runs; when you need to open four files to read one test, the hierarchy is too deep.

2. What lives in the base test

The last row is the one the bench breaks, on purpose, so you can see the cost. run_job is a multi-step job written as a test task. dma_subsys_test extends uvm_test, not dma_base_test (its env is a different class), so it could not call run_job and wrote its own program_dma. Had the job been a sequence that takes an APB sequencer handle, both test classes would start the same object on different sequencers:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

// snippet
class dma_prog_seq extends uvm_sequence #(apb_item);
  `uvm_object_utils(dma_prog_seq)
  bit [31:0] key;
  int unsigned len;
  bit irq_en = 1;

  function new(string name = "dma_prog_seq");
    super.new(name);
  endfunction

  task body();
    write(KEY, key);
    write(LEN, len);
    write(CTRL, {30'b0, irq_en, 1'b1});
  endtask

  task write(bit [7:0] addr, bit [31:0] data);
    apb_rw_seq s = apb_rw_seq::type_id::create("s");
    s.addr = addr; s.write = 1; s.data = data;
    s.start(m_sequencer, this);                 // same sequencer, this as parent
  endtask
endclass

// block test:      prog.start(env.apb.sqr);
// subsystem test:  prog.start(env.dma1.apb.sqr);  prog.start(env.dma0.apb.sqr);

3. Names that say what failed

Use one pattern and enforce it in review: <block>_<feature>_<variant>_test, for example dma_err_truncate_test, dma_stress_backpressure_test. The bench's names, dma_<what>_test, are the short form of the same idea. A name is good if a failure line in a regression report tells you which feature to look at without opening the file. Avoid names that describe history (dma_new_rand_test, dma_rand2_test) or a person; neither tells anyone what is tested.

4. Regression lists are data, separate from the tests

Which tests run, with how many seeds, and with which plusargs, is a list, kept next to the bench and reviewed like code. Tiers keep the feedback fast:

# tier      test                          seeds  plusargs
smoke       dma_smoke_test                1
smoke       dma_rand_test                 5
nightly     dma_rand_test                 200    +DMA_JOBS=10
nightly     dma_err_truncate_test         20
nightly     dma_stress_backpressure_test  50     +UVM_VERBOSITY=UVM_LOW
nightly     dma_stress_reset_test         50
weekly      dma_rand_test                 2000   +DMA_JOBS=50
directed    dma_timeout_test              1      # expects PH_TIMEOUT: judged by a rule, not by "0 fatals"

The smoke tier runs on every commit and must finish in minutes. The nightly tier is where random tests earn their seeds. The last line matters: a test that is supposed to end in a fatal needs its own pass rule in the regression tool, or it either fails every night or teaches people to ignore fatals.

5. New test, new sequence, new plusarg, or new config field?

Watch it run

A healthy library leaves a fingerprint in every log: the test name, the knobs it ran with, and proof of checking. With the base test from Base Test Pattern and the knob logging from +UVM_TESTNAME and Plusargs, one random run reads like this (values depend on the seed):

UVM_INFO @ 0: reporter [RNTST] Running test dma_rand_test...
UVM_INFO ... @ 0: uvm_test_top [KNOBS] n_jobs=3 max_len=16
UVM_INFO dma_tb.sv(731) @ 0: uvm_test_top [JOB] key=90fdd8fd len=10 irq_en=1
...
UVM_INFO dma_tb.sv(411) @ 1305000: uvm_test_top.env.sb [SB] 14 words matched, 0 mismatched
UVM_ERROR :    0
UVM_FATAL :    0

The RNTST, KNOBS and JOB lines are the replay bundle in the log itself; the SB line is proof of work. A regression tool can parse all four.

What breaks: the copied test that ran its parent

An engineer needs a variant of the truncate test that cuts at beat 5. They copy dma_truncate_test into a new file, rename the class to dma_trunc5_test, change the beat, and add it to the nightly list. It passes every night. Weeks later, a bug at beat 5 escapes.

The copied class still said uvm_component_utils(dma_truncate_test). The macro registers whatever name it is given, so the new class never registered itself, and its get_type_name() returned the old name. Selecting +UVM_TESTNAME=dma_trunc5_test would have failed with INVTST, but the list had been edited in a hurry with the old name, so the nightly run executed the original test under the old name and nobody noticed the duplicate. Two defences close this: a lint rule that the class name and the macro argument match, and a family base, so a variant is five lines that set a value instead of a copied file that can drift.

Next: with the environment and the tests designed, the bench needs its judges. Checking & Coverage builds the scoreboard, the predictor and the coverage model on the same bench.

Related topics

• Environment Architecture

• Scenario Inheritance

• Registration macros

• Virtual sequences without a virtual sequencer