Debug playbook

Debug Playbook for Ordering & Outstanding Rules focuses on reorder violation count, outstanding depth, completion latency spread. The goal is to connect the observable symptom to protocol mechanism, ownership, and regression risk.

Protocol debug is a search for the FIRST deviation, not the loudest symptom. Timeouts and error flags are usually many cycles downstream of the real cause.

Root-cause tree

ROOT-CAUSE TREE — Ordering & Outstanding Rules

reorder violation count, outstanding depth, completion latency spread looks wrong
        |
   reproducible?
     /        \
   no          yes
   |            |
 flaky env   same first transaction every time?
 / seed         /            \
              yes             no
               |               |
        protocol rule     timing/reset/PVT
        or config bug     or load-dependent

1. Freeze the failing seed, firmware tag, and spec revision.

2. Find the first bad transaction, not the loudest downstream timeout.

3. Map the transaction to signals and VIP monitor events.

4. Classify the failure: protocol rule, integration config, timing/reset, or performance pressure.

5. Prove the mechanism with one reduced sequence.

6. Patch the smallest owner-controlled boundary and rerun compliance plus workload traffic.

Review memo template

STAFF PROTOCOL REVIEW MEMO — Protocol Fundamentals / Ordering & Outstanding Rules

1. Symptom
   - Watched metric: reorder violation count, outstanding depth, completion latency spread
   - Failing interface: <master/slave/endpoint/controller/PHY>
   - Transaction identity: <ID/tag/address/endpoint/lane>
   - Repro setup: <sim/emulation/FPGA/silicon + firmware tag>

2. Mechanism hypothesis
   - Primary mechanism: IDs, tags, barriers, fences, and completion rules allow concurrency without breaking programmer-visible ordering.
   - Competing hypothesis: <timing, reset, bridge, ordering, firmware, or VIP issue>
   - Missing evidence: <waveform, analyzer trace, counter, spec clause, or log>

3. Proposed action
   - Minimal reversible change: <RTL, register setting, bridge config, scheduler, VIP check>
   - Expected metric movement: <delta and workload>
   - Regression risk: ordering, compatibility, performance, power, area, or timing

4. Signoff
   - Re-run artifact: ID scoreboard, ordering matrix, litmus-style protocol sequence
   - Required owners: architecture owner, RTL owner, software owner
   - Final decision: fix, waive, document limitation, or escalate to architecture

Protocol deep dive

Before naming AXI or PCIe, engineers must master layering, handshakes, ordering, and bandwidth math. These four ideas explain 80% of integration bugs.

Concept diagram

FUNDAMENTALS STACK

software intent
     |
transaction (ID, addr, len, attr, order)
     |
link/channel (handshake, credit, retry)
     |
physical (clock, reset, lanes, PHY)

Debug golden rule: never change layers without carrying transaction identity.

Metric graph

STALL BREAKDOWN EXAMPLE

ready stalls      ████████████████  42%
credit wait       ██████████        26%
ordering block    ██████            16%
reset/config      ████              10%
other             ██                6%

If ready stalls dominate, widening the bus will not help.

Metrics and artifacts to collect

• transaction latency by class

• ready stall cycles

• outstanding depth utilization

• payload efficiency vs headline width

• retry and error rate

Mini case study

A team widened a 64-bit interface to 128-bit but throughput rose only 8% because ready stalls from a slow slave dominated. Fixing slave acceptance and FIFO depth moved the metric; width did not.

Debug branches

• If latency spikes but bandwidth flat, check outstanding limits and ordering.

• If throughput collapses at high load, draw the knee curve — you are past queue stability.

• If intermittent, compare reset release order and clock domain boundaries.

Senior review question

Ask: what is the first transaction that deviates, and which spec rule does it test?

Key takeaways

• Connect every protocol claim to a transaction identity and measurable metric.

• Store the artifact (waveform, log, counter) next to every signoff decision.

Common pitfalls

• Debugging timeouts without finding the first bad transaction.

• Quoting peak bus width without payload efficiency and retry overhead.

• Treating VIP compliance as a substitute for system integration replay.

Principal review addendum

Re-read Ordering & Outstanding Rules against one concrete product workload, not a synthetic directed test.

IDs, tags, barriers, fences, and completion rules allow concurrency without breaking programmer-visible ordering.