CXL Attach Modes: Step-by-Step Walkthrough

Step-by-step analysis walkthrough

Follow this when you own CXL Attach Modes in a protocol review or bring-up war room.

1. State expected transaction in plain language (who initiates, what completes).

2. Draw layer stack and mark clock/reset boundaries.

3. List channels: request, data, response, snoop, credit, or lane.

4. Tag ID/address/endpoint on the failing run.

5. Find first cycle where progress stops or semantics change.

6. Check bridge: width, ID remap, burst, ordering attributes.

7. Check flow control: ready, credit, FIFO, link state.

8. Check firmware/register mode vs hardware capability.

9. Build minimal replay; confirm legal vs illegal per spec.

10. Estimate metric delta from proposed fix.

11. Run compliance + product traffic regression matrix.

12. Write signoff memo with owners and artifacts attached.

Artifacts to collect

• CXL mode table, coherency trace, memory window map

• VIP transaction log

• Waveform with annotations

• Spec clause reference

• Regression manifest

Decision memo template

PROTOCOL DECISION MEMO — CXL Attach Modes
metric:
transaction id:
layer:
hypothesis:
experiment:
fix:
validation:
owners: system architect, CXL controller owner, software owner

Reference visuals

CXL.io / CXL.cache / CXL.mem

CXL SUB-PROTOCOLS (share PCIe PHY)

CXL.io   : PCIe-like discovery, config, DMA            (Type 1/2/3)
CXL.cache: device caches host memory coherently        (Type 1/2)
CXL.mem  : host accesses device-attached memory         (Type 2/3)

Type 3 example: memory expander = mostly CXL.mem + CXL.io.

Protocol deep dive

PCIe is reliable packet delivery over unreliable links; debug flows PHY -> DLL -> TLP -> firmware.

Concept diagram

PCIe DEBUG TOP-DOWN

L0 link healthy?  -> credits OK?  -> TLP completes?  -> driver happy?

Skip a layer and you will mis-own the bug.

Metric graph

LINK DEGRADE EXAMPLE

target x4 Gen4  ---- ---- ---- ----
actual   x4 Gen4  ---- ---- ---- ----   (eval board)
actual   x1 Gen3  -                   (product board)

Package/SI often shows up as width downgrade, not hard fail.

Metrics and artifacts to collect

• link width/speed

• replay count

• completion timeout

• AER error log

• LTSSM history

Mini case study

Endpoint enumerated but DMA timed out: completion credits exhausted because a switch port was misconfigured in firmware, not because the endpoint was broken.

Debug branches

• If degrade at width/speed, PHY/SI before driver.

• If replay storm, link layer before transaction layer.

• If CXL coherency bug, separate .io vs .cache vs .mem traffic.

Senior review question

Ask: what is the first transaction that deviates, and which spec rule does it test?

Key takeaways

• Connect every protocol claim to a transaction identity and measurable metric.

• Store the artifact (waveform, log, counter) next to every signoff decision.

Common pitfalls

• Debugging timeouts without finding the first bad transaction.

• Quoting peak bus width without payload efficiency and retry overhead.

• Treating VIP compliance as a substitute for system integration replay.

Principal review addendum

Re-read CXL Attach Modes against one concrete product workload, not a synthetic directed test.

CXL.io, CXL.cache, and CXL.mem layer coherent and memory semantics on PCIe infrastructure.