Pitfalls and red flags

Pitfalls & Red Flags for CHI Topology Basics focuses on request retry rate, directory occupancy, p99 fabric latency. The goal is to connect the observable symptom to protocol mechanism, ownership, and regression risk.

• A channel looks idle, but upstream credit or ready behavior is the actual blocker.

• The spec allows behavior that the local scoreboard assumed was illegal.

• A bridge changes width, ID, burst shape, or ordering attributes silently.

• Reset releases one side of the interface earlier than the other.

• The fix improves a directed test but regresses real mixed traffic.

Layer responsibility check

LAYER RESPONSIBILITY — CHI Topology Basics

layer          owns                         common failure
-----------    --------------------------   -----------------------
software       intent, ordering needs       wrong assumption
transaction    id/addr/len/attributes       ordering / outstanding
link/channel   handshake, credits, retry    backpressure / deadlock
physical       clock/reset/lanes/PHY        timing / training / SI
observability  waveform/log/counter         missing evidence

Protocol deep dive

Coherence extends memory transactions with snoop and state — traffic multiplies when software shares cache lines.

Concept diagram

COHERENCE TRAFFIC FLOW

RN issues coherent read
   -> HN looks up directory
   -> snoops to sharers
   -> data + state update returned

False sharing: different variables, same cache line -> coherence storm.

Metric graph

COHERENCY TRAFFIC STACK

data fetch        ████████
snoop responses   ██████████████
writebacks        ██████
maintenance ops   ████

High snoop stack with good IPC -> suspect line sharing before faster NoC.

Metrics and artifacts to collect

• snoop rate

• intervention latency

• coherency transaction mix

• false sharing indicators

Mini case study

Benchmark IPC looked fine but system power spiked: per-core counters were on one cache line. Padding counters fixed coherency traffic without any NoC change.

Debug branches

• If snoop latency high, check home node placement and directory policy.

• If ordering bug, run litmus sequences before microarch changes.

• If traffic storm, profile cache line sharing in software layout.

Senior review question

Ask: what is the first transaction that deviates, and which spec rule does it test?

Key takeaways

• Connect every protocol claim to a transaction identity and measurable metric.

• Store the artifact (waveform, log, counter) next to every signoff decision.

Common pitfalls

• Debugging timeouts without finding the first bad transaction.

• Quoting peak bus width without payload efficiency and retry overhead.

• Treating VIP compliance as a substitute for system integration replay.

Principal review addendum

Re-read CHI Topology Basics against one concrete product workload, not a synthetic directed test.

CHI separates request, response, data, and snoop flows across nodes with directory and home-node responsibilities.