Little's law sizes every queue: average outstanding equals throughput times average latency. If the product exceeds the queue depth, the queue saturates and latency climbs.
Memory-level parallelism (MLP) is the outstanding count the core sustains. Average memory access time improves with MLP because overlapped miss penalties divide by the overlap factor.
Little check on the hub waterfall (illustrative, 1 GHz core):
request rate = 118 GB/s / 64 B per request = 1.84375e9/s ~ 1.8 billion/s
latency = 43 cycles at 1 GHz = 43 ns
needed = 118 x 43 / 64 = 5074 / 64 = 79.28 ~ 79 outstanding
queue has 64 slots, and 79 > 64, so the controller queue saturates.
AMAT with MLP: 0.05 miss rate x 200-cycle penalty / 4-way MLP = 2.5 cycles added.Measure delivered bandwidth at each hop under one synchronized window. The first hop whose queue latency spikes with demand is the bottleneck.
Hop | Delivered (GB/s) | Loss (GB/s) | Share of loss |
|---|---|---|---|
Core demand | 164 | — | — |
LLC | 141 | 23 | 50% |
NoC | 123 | 18 | 39% |
Controller and DRAM | 118 | 5 | 11% |
Loss shares use rounding and sum to 100 percent. Efficiency is 118/164, about 72 percent, matching the hub hook.
The model below checks waterfall arithmetic and the Little's-law saturation test; it is accounting, not a DRAM timing model.
module bw_waterfall #(
parameter int OFFERED = 164,
parameter int LOSS_LLC = 23,
parameter int LOSS_NOC = 18,
parameter int LOSS_CTRL = 5,
parameter int REQ_BYTES = 64,
parameter int LAT_NS = 43,
parameter int QUEUE_SLOTS = 64
) (
output logic [31:0] delivered,
output logic [31:0] eff_permille,
output logic [31:0] need_outstanding,
output logic saturated
);
always_comb begin
delivered = OFFERED - LOSS_LLC - LOSS_NOC - LOSS_CTRL;
eff_permille = (delivered * 1000) / OFFERED;
need_outstanding = (delivered * LAT_NS) / REQ_BYTES;
saturated = (need_outstanding > QUEUE_SLOTS);
end
endmodule
module tb_waterfall;
logic [31:0] delivered, eff_permille, need_outstanding;
logic saturated;
int errors = 0;
bw_waterfall dut (.*);
initial begin
#1;
if (delivered !== 32'd118) errors++;
if (eff_permille !== 32'd719) errors++;
if (need_outstanding !== 32'd79) errors++;
if (saturated !== 1'b1) errors++;
if (errors == 0) $display("PASS: waterfall delivers 118 of 164 GB/s with 79 outstanding over 64 slots");
else $display("FAIL: %0d waterfall checks wrong", errors);
$finish;
end
endmodule118 × 43 / 64 is about 79. A 64-slot queue cannot hold them, so it saturates.
It halves, from 2.5 cycles to 1.25 cycles. Overlap divides the penalty term.