Reproduce the exact overlap

Simulation deadlocks only when GPU DMA bursts overlap CPU coherent traffic. Either stream alone drains, so the bug lives in their interaction.

The offered load tells the story. Assume one 32 GB/s link, 20 GB/s of coherent traffic and 25 GB/s of DMA bursts: 45 GB/s offered against 32 GB/s of capacity.

1. 0-5 min: capture the traffic mix and the deadlock signature.

2. 5-15 min: list topology, routing rule, buffer classes and credit loops.

3. 15-25 min: draw the wait cycle or the starvation pattern.

4. 25-35 min: propose class separation plus QoS or routing fixes.

5. 35-45 min: prove the fix with a deadlock check and a regression run.

Classes prevent protocol deadlock

A request that cannot sink blocks its buffer. If a response needs that same buffer to complete the request, the pair waits forever.

Virtual channels break routing deadlock only. Protocol deadlock needs separate buffer classes for requests and responses, so responses always drain.

Size buffers with Little's law

Each 64-byte line travels as four 16-byte flits. Buffers must cover the credit round trip at the injection rate.

Little's law gives outstanding flits as rate times latency: 0.25 x 20 = 5 flits, so provision 8 buffers. The credit model below enforces the same rule.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

module credit_mgr (
  input  logic       send,
  input  logic       giveback,
  input  logic [7:0] count_in,
  output logic [7:0] count_out,
  output logic       denied
);
  parameter int MAXC = 8;
  always_comb begin
    denied = send && !giveback && (count_in == 8'd0);
    count_out = count_in;
    if (send && !giveback) begin
      if (count_in != 8'd0) count_out = count_in - 8'd1;
    end else if (giveback && !send) begin
      if (count_in < MAXC) count_out = count_in + 8'd1;
    end
  end
endmodule

The testbench proves credit discipline. Five sends move 8 to 3, two returns move 3 to 5, five sends drain to 0, one more send is refused, and three returns restore 3.

Fix with classes plus QoS

Split request and response buffers first, then cap the bully. Capping DMA at 10 GB/s leaves 20 + 10 = 30 GB/s against 32 GB/s of capacity.

Offered: 20 + 25 = 45 GB/s against a 32 GB/s link, backlog grows
Capped:  20 + 10 = 30 GB/s against a 32 GB/s link, backlog drains
Buffers: 0.25 flits per cycle x 20 cycles = 5 flits, provision 8

Related topics

• On-chip networks

• Performance method