Extended case study

Integration review: NACK rate, clock-stretch duration, bus recovery time regresses after a change touching I2C Timing & Arbitration.

Background

Baseline traffic passed compliance and performance targets. A bridge update, firmware change, or clock/reset tweak introduced intermittent failures visible only under mixed traffic.

Symptoms observed

• Regression in NACK rate, clock-stretch duration, bus recovery time

• VIP warning followed by software timeout (symptom lag)

• Directed tests pass; stress or product replay fails

• Two teams disagree because they look at different layers

Investigation timeline

1. Hour 0: freeze sim tag, firmware, and spec revision

2. Hour 1: capture first failing transaction with ID/address

3. Hour 2: correlate waveform, VIP monitor, and counter

4. Hour 3: classify: rule violation vs config vs timing vs load

5. Hour 4: reduce to 3-transaction minimal sequence

6. Hour 5: bounded RTL or register fix + regression list

7. Hour 6: compliance replay + product workload signoff memo

Root cause

The failing behavior traced to a violated assumption in I2C Timing & Arbitration: I2C shares open-drain wires with address arbitration, acknowledge cycles, and optional clock stretching.

Fix and validation

• Minimal reversible change at the owning boundary

• Re-run SCL/SDA waveform, transaction log, pull-up timing check on failing and baseline seeds

• Compliance suite + mixed-traffic regression

• Document software-visible impact and waiver if any

Lessons learned

• First bad transaction beats loudest timeout

• Layer alignment across RTL, VIP, firmware, and analyzer

• Performance and correctness regressions need separate evidence

CASE STUDY METRICS — I2C Timing & Arbitration

baseline     NACK rate, clock-stretch duration, bus recovery time: within target
regressed    NACK rate, clock-stretch duration, bus recovery time: fails product threshold
after fix    NACK rate, clock-stretch duration, bus recovery time: restored + compliance PASS
residual risk: document waiver or monitor in field

Sequence under stress

SEQUENCE — I2C Timing & Arbitration

  initiator            interconnect/PHY            target
      |  request (id) ------->  |                     |
      |                         |  forward ----------> |
      |                         |                     | work
      |                         |  <---- response ---- |
      |  <----- complete ------ |                     |
      |
   metric captured here: NACK rate, clock-stretch duration, bus recovery time

Protocol deep dive

I2C/SPI/UART bugs are contract bugs: timing, reset value, IRQ type, and DMA watermark.

Concept diagram

PERIPHERAL CONTRACT

firmware writes regs -> RTL state machine -> pins -> board -> device
        ^                    |
        +------- IRQ/DMA ----+

If IRQ is level but driver assumes edge, you get lost events.

Metric graph

FIFO WATERMARK vs DMA

FIFO fill
 100%|        *** overrun risk
  75%|     ***
  50%|  ***     <- ideal DMA trigger band
  25%| *
   0%+----------------> time

Metrics and artifacts to collect

• NACK rate

• overrun count

• CS setup/hold violations

• IRQ miss rate

Mini case study

SPI flash worked in loopback but failed in system: CS deasserted one cycle early relative to device hold time. Board + RTL + mode bits together formed the contract.

Debug branches

• If overrun, FIFO depth vs ISR latency vs DMA burst.

• If NACK on I2C, pull-ups, speed, and clock stretch.

• If garbage data, CPOL/CPHA and MSB/LSB first.

Senior review question

Ask: what is the first transaction that deviates, and which spec rule does it test?

Key takeaways

• Connect every protocol claim to a transaction identity and measurable metric.

• Store the artifact (waveform, log, counter) next to every signoff decision.

Common pitfalls

• Debugging timeouts without finding the first bad transaction.

• Quoting peak bus width without payload efficiency and retry overhead.

• Treating VIP compliance as a substitute for system integration replay.

Field case notes

Mixed traffic exposed a bug that single-master directed tests missed for three weeks.