Root cause: the interrupt crosses into the FSM clock with no synchroniser and fans out to two destinations, the next-state logic and a status register. When the interrupt changes inside the capture window, the two loads resolve differently often enough over weeks of operation that the FSM eventually sees a next-state input inconsistent with its current state and jumps somewhere illegal. The bench never sees it because the alignment is rare, and simulation never sees it because ideal clocks have no setup-hold window.

The fix

1
2
3
4
5
6
7
8

// snippet: legal single-bit level synchroniser before the fanout
// The interrupt must be a level held stable across the crossing;
// pulses need a toggle or handshake scheme instead.
always @(posedge clk_slow) begin
  sync1 <= irq_fast;   // first capture flop: may go metastable
  sync2 <= sync1;      // second flop: samples a settled value
end
// All FSM and status fanout is driven from sync2 ONLY.

Two rules make the fix real. First, every load of the interrupt comes from the synchroniser output; the raw crossing drives nothing. Second, the interrupt must be a level held stable, not a pulse: if the source only pulses, the fix is a toggle synchroniser or handshake instead, per Why Pulses Need Help. A fix that synchronises a pulse with two flops trades the illegal state for a lost event.

The closing evidence

A re-run structural analysis showing the crossing synchronised with no logic between flops, a metastability-aware simulation proving the FSM tolerates either resolution, and a review note recording the fanout rule so the next engineer cannot tap the raw crossing again. Simulation is supporting evidence here, not proof: say so out loud.

Related topics

• Next: How to Narrate It

• RTL: Metastability and Synchronisation