A new engineer joins the team and reads the regression list: dma_smoke_test, dma_rand_test, dma_truncate_test, dma_drain_test, dma_timeout_test, dma_vseq_test, dma_backpressure_test, dma_reset_test, dma_irq_agent_test. They all send packets through the same DMA. Their question is a fair one: what does each of these prove that the others do not?

If a test cannot answer that in one sentence, it is either redundant or doing too much. This lesson reads each derived test as a scenario, sorted by what it changes, and shows the three shapes a good derived test takes.

The idea: a derived test changes one kind of thing

A derived test inherits the whole set from the base test and changes something. On the bench there are only three kinds of change. Stimulus: the test writes a run_phase that drives a different job. Behaviour: the test registers a factory override in build_phase so that a component behaves differently (a driver that truncates, a scoreboard that owns end of test), and then drives a job that exercises it. Policy: the test changes a rule of the run itself, such as the timeout.

Recipe variations work the same way. "Pancakes, but with buckwheat flour" changes an ingredient; "pancakes, but in a cast-iron pan" changes the equipment; "pancakes, but stop after ten minutes whatever happens" changes the rule. Each variation names its one change. A variation that changes the flour, the pan and the timer at once is a different recipe, and deserves a different name.

The picture: the nine tests by what they change

Build it: the three shapes

Stimulus only: dma_rand_test

The simplest derived test has a constructor, a registration macro and a run_phase. dma_rand_test raises its objection, runs n_jobs jobs through the base test's run_job, checks STATUS after each, and drops the objection. Each job is a dma_job_cfg object: a randomizable bundle of KEY, LEN and IRQ_EN with constraints that keep it legal.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

// snippet: from the course DMA bench
// Several random jobs. A job config placed in config_db by a derived test or the
// command line wins; otherwise each job is randomized here.
class dma_rand_test extends dma_base_test;
  `uvm_component_utils(dma_rand_test)
  int unsigned n_jobs = 3;

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  task run_phase(uvm_phase phase);
    dma_job_cfg job;
    bit [31:0] status;
    phase.raise_objection(this);
    for (int j = 0; j < n_jobs; j++) begin
      if (!uvm_config_db#(dma_job_cfg)::get(this, "", "job", job)) begin
        job = dma_job_cfg::type_id::create($sformatf("job%0d", j));
        if (!job.randomize()) `uvm_fatal("RAND", "dma_job_cfg randomize failed")
      end
      `uvm_info("JOB", job.convert2string(), UVM_LOW)
      run_job(job, job.len, status);
      if (status[2:1] !== 2'b01) `uvm_error("STATUS", $sformatf("job %0d: STATUS=%08h, expected DONE only", j, status))
    end
    #100ns;
    phase.drop_objection(this);
  endtask
endclass

Notice the get at the top of the loop. If something put a dma_job_cfg named job in config_db for this test, it is used instead of a random one. That gives a debug path, pinning a failing job without editing the test, and it is also the source of the bug in What breaks below. Randomizing an object with randomize() with { ... } is how a test aims stimulus without touching the sequence class; Configuring Sequences covers the other ways to hand knobs to stimulus.

Override plus stimulus: dma_truncate_test

A test that exercises a new behaviour needs two halves: something that creates the condition, and a job that runs into it. dma_truncate_test swaps the stream driver for axis_truncating_driver, which forces TLAST early, and then runs an 8-word job and checks that the DMA reported ERR.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// snippet: from the course DMA bench
// Factory override: the stream driver cuts each packet short, so the DMA must
// stop early and report ERR. No sequence or env file changes.
class dma_truncate_test extends dma_base_test;
  `uvm_component_utils(dma_truncate_test)

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  function void build_phase(uvm_phase phase);
    axis_driver::type_id::set_type_override(axis_truncating_driver::get_type());
    super.build_phase(phase);                // the env is created after the override
  endfunction

  task run_phase(uvm_phase phase);
    dma_job_cfg job = dma_job_cfg::type_id::create("job");
    bit [31:0] status;
    phase.raise_objection(this);
    if (!job.randomize() with { len == 8; }) `uvm_fatal("RAND", "dma_job_cfg randomize failed")
    run_job(job, 8, status);
    if (status[2] !== 1'b1) `uvm_error("STATUS", $sformatf("ERR not set after a short packet: STATUS=%08h", status))
    else `uvm_info("TRUNC", "DMA reported ERR for the short packet, as expected", UVM_LOW)
    #100ns;
    phase.drop_objection(this);
  endtask
endclass

No sequence, agent or env file changed. That is the reason overrides exist: the condition is injected from the test, so the env stays the same for every other test. The next lesson goes through the override calls themselves.

Policy: dma_timeout_test

dma_timeout_test changes no component and no stimulus shape. It sets a 20 µs global timeout and runs a job with the interrupt disabled, so run_job waits forever and the timeout ends the run with a fatal. Its purpose is to prove the safety net works; run_test and timeouts is the home of the mechanism.

Why not default_sequence?

UVM can start a sequence automatically on a sequencer in a given phase if you put its type in config_db under default_sequence. The bench never does, and most teams avoid it for tests: the stimulus is then invisible in the test's code, it is tied to a phase name, and it needs a sequencer to hang on (the bench's virtual sequence has none). Starting a sequence explicitly in run_phase keeps the scenario readable in one place.

Watch it run

The random test logs each job before running it. The values come from the seed and differ by simulator:

UVM_INFO dma_tb.sv(731) @ 0: uvm_test_top [JOB] key=90fdd8fd len=10 irq_en=1
UVM_INFO dma_tb.sv(731) @ 645000: uvm_test_top [JOB] key=0e564b57 len=1 irq_en=1
UVM_INFO dma_tb.sv(731) @ 885000: uvm_test_top [JOB] key=5a1793f4 len=3 irq_en=1
UVM_INFO dma_tb.sv(411) @ 1305000: uvm_test_top.env.sb [SB] 14 words matched, 0 mismatched
UVM_ERROR :    0

• 10 + 1 + 3 = 14 words. The scoreboard's count should always equal the sum of the job lengths; check it when you read a log.

• irq_en=1 on every job comes from a soft constraint in dma_job_cfg. A derived test can override it with with { irq_en == 0; } because it is soft.

• Logging the job before running it is what makes a failing seed easy to reproduce: the log already holds the exact values.

What breaks: the repro test that repeated one job three times

Seed 4417 of dma_rand_test fails on its first job. To debug it without the seed, an engineer pins that job in a small derived test:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// snippet
class dma_rand_repro_test extends dma_rand_test;
  `uvm_component_utils(dma_rand_repro_test)

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  function void build_phase(uvm_phase phase);
    dma_job_cfg job = dma_job_cfg::type_id::create("job");
    super.build_phase(phase);
    job.key = 32'h90fd_d8fd; job.len = 10; job.irq_en = 1;
    uvm_config_db#(dma_job_cfg)::set(this, "", "job", job);
  endfunction
endclass

The failure reproduces, a fix goes in, and the repro test joins the nightly list as a regression guard "for the first job, with two random jobs after it". Months later someone notices it never randomizes anything. The get in the loop succeeds on every iteration and returns the same handle, so all three jobs are the pinned one. The test was correct for its first purpose and misnamed for its second.

The fix is to say what the test does. Either set n_jobs = 1 in the repro test so it runs exactly the pinned job, or change the base loop to take the pinned job only for iteration 0. The general rule: when a test inherits a loop, check what the loop does with your configuration on every pass, not only the first.

Next: five of the nine tests start with a factory override. Factory Overrides in Tests shows how each one is aimed, and the one case where the order of two lines decides whether the test works at all.

Related topics

• Factory Overrides in Tests

• Designing a Test Library

• Configuring sequences

• run_test and timeouts