A link was sized carefully: 12 receiver credits against a 10-cycle credit round trip, comfortably full throughput. To save bandwidth on the return path, a later change sends credits back in groups of eight instead of one at a time. Throughput drops to about 60%, and the change was "only a return-path optimisation".

How much does returning credits late, or in batches, cost, and when is it worth it?

The obvious answer, and why it breaks

The obvious view is that credit returns are bookkeeping: sending them in batches saves messages and changes nothing important, because the credits all arrive eventually. But lesson 4.5 showed that a credit is useless while it is not at the sender. A credit that waits at the receiver for its batch to fill is, for that time, still "in the loop". Batching lengthens the round trip of every credit by up to the time it takes to fill the batch.

The idea: the loop includes every cycle a credit waits

Count a credit's round trip including the time it sits at the receiver waiting to be returned. With batches of K, a freed credit may wait for up to K − 1 more slots to free before the batch goes out. At one word per cycle, the effective loop grows by up to K − 1 cycles, so full throughput now needs:

With fewer credits than that, throughput falls below one word per cycle. When B is not a multiple of K, there is a further loss: up to K − 1 credits can be stranded at the receiver, waiting for a batch that the remaining credits are too few to complete quickly.

Build it: batch size against throughput

The model is lesson 4.5's credit loop with 4 stages each way (round trip 10 cycles) and 12 credits, so batches of one give full throughput. It then returns credits only in groups of K and measures steady-state throughput.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

"""Credit return batching: return credits in groups of K instead of one at a time.

Same loop as lesson 4.5: L = 4 stages each way, receiver buffer B = 12 (the round
trip without batching is 2L + 2 = 10 cycles, so 12 credits give full throughput).
Now the receiver holds freed credits until it has K of them, then returns all K at once.
"""
from collections import deque


def simulate(L, B, K, cycles=6000):
    credits, buf, pending, sent_late = B, 0, 0, 0
    fwd, back = deque([0] * L), deque([0] * L)
    for t in range(cycles):
        arrive, crd_in = fwd[-1], back[-1]
        send = 1 if credits > 0 else 0
        pop = 1 if buf > 0 else 0
        pending += pop
        ret = 0
        if pending >= K:
            ret, pending = K, pending - K
        credits += crd_in - send
        buf += arrive - pop
        fwd.rotate(1); fwd[0] = send
        back.rotate(1); back[0] = ret
        assert credits + sum(fwd) + buf + pending + sum(back) == B
        if t >= cycles // 2:
            sent_late += send
    return sent_late / (cycles - cycles // 2)


BATCH = [1, 2, 4, 6, 8, 12]
row = [simulate(4, 12, k) for k in BATCH]
print("batch size K:  " + " ".join(f"{k:5d}" for k in BATCH))
print("throughput:    " + " ".join(f"{x:5.3f}" for x in row))
assert row[0] == 1.0 and row[-1] < row[0]
print("PASS: batching holds credits back; once B no longer covers RTT + K - 1, throughput falls")

It prints:

batch size K:      1     2     4     6     8    12
throughput:    1.000 1.000 0.923 0.800 0.615 0.572
PASS: batching holds credits back; once B no longer covers RTT + K - 1, throughput falls

Batches of 1 and 2 cost nothing: 12 ≥ 10 + 1. At 4, 12 < 10 + 3 and throughput is 12/13. At 6 it is 12/15 = 0.80. At 8 it falls further than the formula's 12/17, to 0.615, because four credits are stranded: the receiver holds them waiting for a batch of eight. At 12, all credits travel as one batch and the loop is 21 cycles: 12/21 ≈ 0.57.

Worked numbers: return bandwidth versus buffer

Batching exists because credit messages cost bandwidth on the return path. With illustrative numbers:

Forward traffic: one 64-byte flit per cycle; credit message: 8 bytes, carrying a count
Return one credit per message:   8 bytes per cycle on the return path (12.5% of the forward rate)
Return 4 credits per message:    2 bytes per cycle (3.1%), but B must grow by K - 1 = 3 flits
                                 = 3 x 64 = 192 bytes of extra buffer to keep full throughput

The trade is explicit: return bandwidth against buffer. Where the return path also carries other traffic (responses, for example), saving return bandwidth can be worth a few extra buffer entries; where the buffer is SRAM-limited, returning promptly is cheaper.

Other ways credits come back late

• The return path is shared and credit messages wait behind other traffic: give credits priority, or carry them in the headers of reverse-direction packets.

• The receiver frees slots late, for example only after its consumer has fully processed a word: return the credit as soon as the slot itself is free.

• A clock-domain crossing on the return path adds synchroniser cycles to every credit: count them in the round trip.

The design space

How real protocols chose

What breaks

The opening story (illustrative): batching credits by eight lengthened the effective loop from 10 to 17 cycles and stranded four credits, so a link with 12 credits ran at about 60%. The fix was a hybrid many designs use: return credits in batches when the link is busy in both directions, but return any pending credits immediately when the sender is running low or the return path is idle, so batching never starves the sender.

Key takeaways

• A credit's round trip includes any time it waits at the receiver before being returned.

• Batching K credits saves return bandwidth but needs B ≥ RTT + K − 1 for full throughput, and can strand credits.

• Every return-path change belongs in the credit-loop sizing; batch when busy, flush when the sender runs short.

Related topics

• Part 4 overview: Flow Control