A new team member wants to delete the hand-written scoreboard: "UVM already has a comparator, why maintain our own?" They wire uvm_in_order_class_comparator to the predictor and the output monitor and run the smoke test. It reports 8 matches and 0 mismatches. To be sure, they make the predictor deliberately wrong and run it again. 8 matches, 0 mismatches. A checker that cannot fail is worse than no checker, because it looks like one. How did UVM's own comparator get this wrong, and when is a built-in comparator the right tool?

The idea: a comparator is two queues and a rule

UVM ships a small family of comparators. Each has a before_export for expected items and an after_export for actual items. Inside, two analysis FIFOs hold whatever arrives, and a run_phase loop takes one item from each, applies a comparison rule, counts a match or a mismatch, and goes round again. It is a ticket inspector with two queues: one of tickets sold, one of passengers, checked strictly in arrival order.

The comparison rule is a class parameter, a policy: a class with a static comp(a, b) function. A second policy, the converter, turns an item into a string for messages, and a third parameter names the pair type written on the comparator's pair_ap. Choose the policies and you choose what "equal" means.

The picture: what each parameter does

For a class type, the default comp_type compares handles: a == b is true only if both are the same object, which two separately created items never are. So with class items you must always supply a policy, or use the class comparator, which supplies one for you. That is where the trap is.

Build it: the comparator in dma_env_check

The policy is a plain class with one static function. It defines equality as "same data word", which is exactly what the DMA promises.

1
2
3
4
5
6
7

// snippet: from the course DMA bench
// A comparison policy for uvm_in_order_comparator: compare the data field.
class axis_data_comp;
  static function bit comp(axis_item a, axis_item b);
    return a.data == b.data;
  endfunction
endclass

1
2
3
4

// snippet: the comparator types, from the bench package
typedef uvm_in_order_comparator #(axis_item, axis_data_comp, uvm_class_converter #(axis_item),
                                 uvm_class_pair #(axis_item, axis_item)) axis_in_order_comparator;
typedef uvm_in_order_class_comparator #(axis_item) axis_class_comparator;

In dma_env_check (shown in Predictor and Reference-Model Design), the predictor's exp_ap goes to before_export and the output monitor to after_export. On dma_check_test the comparator reports in-order comparator: 14 matches, 0 mismatches, agreeing with the scoreboard.

What breaks: the comparator that agreed with eight wrong predictions

dma_cmp_trap_test is the opening story, built into the bench. It replaces the predictor with dma_predictor_buggy, which uses the inverted key, and asks the env for the class comparator instead of the policy one.

1
2
3
4
5
6
7
8
9
10
11

// snippet: from the course DMA bench
// Used only by the comparator-trap demo: predicts with the wrong key.
class dma_predictor_buggy extends dma_predictor;
  `uvm_component_utils(dma_predictor_buggy)
  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction
  function bit [31:0] predict(bit [31:0] data);
    return data ^ ~key;
  endfunction
endclass

1
2
3
4
5
6
7
8
9
10
11
12
13

// snippet: from the course DMA bench
class dma_cmp_trap_test extends dma_smoke_test;
  `uvm_component_utils(dma_cmp_trap_test)
  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction
  function void build_phase(uvm_phase phase);
    dma_env::type_id::set_type_override(dma_env_check::get_type());
    dma_predictor::type_id::set_type_override(dma_predictor_buggy::get_type());
    uvm_config_db#(bit)::set(this, "env", "use_class_cmp", 1);
    super.build_phase(phase);
  endfunction
endclass

+UVM_TESTNAME=dma_cmp_trap_test
UVM_INFO dma_tb.sv(411) @ 655000: uvm_test_top.env.sb [SB] 8 words matched, 0 mismatched
UVM_INFO dma_tb.sv(1637) @ 655000: uvm_test_top.env [CMP] class comparator: 8 matches, 0 mismatches
UVM_ERROR :    0
UVM_FATAL :    0

The DUT is correct, so the scoreboard, which has its own correct model, matches all 8 words. Every one of the predictor's 8 expected items is wrong, and the class comparator calls all 8 a match. Follow the call: uvm_in_order_class_comparator uses the policy uvm_class_comp, whose comp(a, b) returns a.compare(b). compare() asks the comparer, which calls the item's do_compare() and compares any fields registered with field macros. axis_item registers no fields and has no do_compare, so the base uvm_object::do_compare runs, and it returns 1. Nothing was compared; everything matched.

The fix: give the item a real do_compare

Compare exactly the fields a monitor observes. idle is a driver knob, not something on the bus, so it stays out.

1
2
3
4
5
6

// snippet: the method axis_item needs before any class-based comparator can trust it
function bit do_compare(uvm_object rhs, uvm_comparer comparer);
  axis_item r;
  if (!$cast(r, rhs)) return 0;
  return super.do_compare(rhs, comparer) && data == r.data && last == r.last;
endfunction

A scratch run (Verilator) gave axis_item this method through a derived class and a factory type override, and reran the trap. The class comparator now reports 0 matches, 8 mismatches:

UVM_ERROR @ 205000: uvm_test_top.env.class_cmp [Comparator Mismatch] data=ae3ce6b4      differs from data=51c3194b
...
UVM_INFO dma_tb.sv(1637) @ 625000: uvm_test_top.env [CMP] class comparator: 0 matches, 8 mismatches
UVM_ERROR :    8

Those lines say UVM_ERROR only because the scratch test asked for it. Out of the box, the comparator reports a mismatch with uvm_report_warning, ID Comparator Mismatch. A regression that greps for UVM_ERROR : 0 passes a run full of comparator mismatches. Promote the ID on the comparator, once, in the env or the base test:

// snippet: make comparator mismatches fail the test
cmp.set_report_severity_id_override(UVM_WARNING, "Comparator Mismatch", UVM_ERROR);

Two more gaps are the comparator's, not your item's. It has no end-of-test check, so expected items still waiting in its FIFO when the test ends are never reported; and its FIFOs are local, so you cannot inspect them. Count the expected items you publish and, in check_phase, fail if m_matches + m_mismatches falls short. Its flush() resets only the two counters, not the FIFOs.

The algorithmic comparator: a transformer instead of a predictor

uvm_algorithmic_comparator #(BEFORE, AFTER, TRANSFORMER) takes raw inputs on before_export and calls transform() on a transformer object to turn each one into an expected item. Its after_export takes the actual items. It is a predictor and a comparator in one box, useful when the model is a pure function of each input.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

// snippet: the DMA as an algorithmic comparator
class dma_xform;
  dma_predictor pred;                       // reuses the predictor's key tracking
  function axis_item transform(axis_item in);
    axis_item e = axis_item::type_id::create("exp");
    e.data = pred.predict(in.data);
    e.last = in.last;
    return e;
  endfunction
endclass

typedef uvm_algorithmic_comparator #(axis_item, axis_item, dma_xform) axis_alg_comparator;

// in the env:  build_phase:   xf = new();  alg = new("alg", this, xf);
//              connect_phase: xf.pred = pred;
//                             in_agt.ap.connect(alg.before_export);
//                             out_agt.ap.connect(alg.after_export);

Two details matter. The transformer is a constructor argument, so the comparator is built with new(), not type_id::create(), and a factory override cannot replace it. And the compare inside it is a uvm_in_order_class_comparator of the AFTER type: the same trap. A scratch run (Verilator) with a correct transformer reported 8 matches; with the buggy predictor behind the transformer it still reported 8 matches, 0 mismatches. Without do_compare on the item, the algorithmic comparator cannot fail either.

When do built-ins beat a hand-written scoreboard?

A built-in comparator is a fine second opinion on an in-order stream, which is how dma_env_check uses it: a cheap, independent cross-check next to the scoreboard. As the only checker it needs a policy (or a trustworthy do_compare), a promoted mismatch ID and your own leftover check, and by then you have written most of a scoreboard.

A tool bug on the way: passing pair_type explicitly

While the bench was being built, Verilator stopped with an internal error on uvm_in_order_comparator #(axis_item, axis_data_comp, uvm_class_converter #(axis_item)), letting pair_type default to uvm_built_in_pair #(axis_item). Naming the fourth parameter explicitly as uvm_class_pair #(axis_item, axis_item) made it compile. That is a tool bug, not a UVM rule, but the explicit form is better anyway: for class items, the class pair is the right type for pair_ap, since its do_compare and do_copy call the items' methods instead of comparing handles.

Next: the checks are now trustworthy, which raises the other half of sign-off: did the tests try everything that matters? That is Coverage & Closure.

Related topics

• Coverage & Closure

• Scoreboard architecture

• Predictor and reference-model design