Extended case study

Integration review: register response latency, stream bubble rate, packet boundary errors regresses after a change touching AXI-Lite & AXI-Stream.

Background

Baseline traffic passed compliance and performance targets. A bridge update, firmware change, or clock/reset tweak introduced intermittent failures visible only under mixed traffic.

Symptoms observed

• Regression in register response latency, stream bubble rate, packet boundary errors

• VIP warning followed by software timeout (symptom lag)

• Directed tests pass; stress or product replay fails

• Two teams disagree because they look at different layers

Investigation timeline

1. Hour 0: freeze sim tag, firmware, and spec revision

2. Hour 1: capture first failing transaction with ID/address

3. Hour 2: correlate waveform, VIP monitor, and counter

4. Hour 3: classify: rule violation vs config vs timing vs load

5. Hour 4: reduce to 3-transaction minimal sequence

6. Hour 5: bounded RTL or register fix + regression list

7. Hour 6: compliance replay + product workload signoff memo

Root cause

TLAST missing on final stream beat caused downstream packet reassembly to wait forever.

Fix and validation

• Minimal reversible change at the owning boundary

• Re-run TVALID/TREADY trace, TLAST packet log, control register access test on failing and baseline seeds

• Compliance suite + mixed-traffic regression

• Document software-visible impact and waiver if any

Lessons learned

• First bad transaction beats loudest timeout

• Layer alignment across RTL, VIP, firmware, and analyzer

• Performance and correctness regressions need separate evidence

CASE STUDY METRICS — AXI-Lite & AXI-Stream

baseline     register response latency, stream bubble rate, packet boundary errors: within target
regressed    register response latency, stream bubble rate, packet boundary errors: fails product threshold
after fix    register response latency, stream bubble rate, packet boundary errors: restored + compliance PASS
residual risk: document waiver or monitor in field

Sequence under stress

SEQUENCE — AXI-Lite & AXI-Stream

  initiator            interconnect/PHY            target
      |  request (id) ------->  |                     |
      |                         |  forward ----------> |
      |                         |                     | work
      |                         |  <---- response ---- |
      |  <----- complete ------ |                     |
      |
   metric captured here: register response latency, stream bubble rate, packet boundary errors

Protocol deep dive

AMBA is the on-chip lingua franca: APB for control, AHB for legacy bursts, AXI for high-performance coherent fabrics.

Concept diagram

AMBA INTEGRATION MAP

CPU --AXI--> NIC --AXI--> SRAM
  |              |
  +--AXI-Lite--> peripherals (GPIO, timers)
  +--AXI-Stream-> video pipe

Every bridge is a contract rewrite: width, ID, burst, cache attrs.

Metric graph

AXI CHANNEL ACTIVITY (mixed traffic)

AW+W     ████████████
AR       ████████████████
R        ██████████████
B        ████████

Read-heavy phase: AR/R dominate; write resp may look idle while system is healthy.

Metrics and artifacts to collect

• AR/AW/R/B channel utilization

• write resp latency

• read OSTD depth

• SLVERR/DECERR count

• bridge hang log

Mini case study

Write burst hung because W beats arrived before AW for a narrow bridge that reordered channels. VIP flagged nothing until full-system traffic interleaved reads and writes.

Debug branches

• Hung write: verify AW/W ordering and wlast alignment.

• Hung read: check arready stall and rlast per ID.

• Decode errors: address map vs interconnect route table.

Senior review question

Ask: what is the first transaction that deviates, and which spec rule does it test?

Key takeaways

• Connect every protocol claim to a transaction identity and measurable metric.

• Store the artifact (waveform, log, counter) next to every signoff decision.

Common pitfalls

• Debugging timeouts without finding the first bad transaction.

• Quoting peak bus width without payload efficiency and retry overhead.

• Treating VIP compliance as a substitute for system integration replay.

Field case notes

Mixed traffic exposed a bug that single-master directed tests missed for three weeks.