A home node in a coherent fabric has 64 tracker entries for transactions in progress. Hundreds of requesters may send it requests at any moment. If requests that find the trackers full wait in the fabric, they hold link buffers and block other traffic, which can include the responses that would free the trackers. If they are rejected instead, they come back, and may be rejected again, for ever. Both simple answers fail.
How should a target handle more requests than it has room for, without deadlock or endless retries?
One obvious answer is to queue: requests wait in the fabric until the target has room. That is hold and wait (lesson 9.2): waiting requests hold buffers, and if what they block includes traffic the target needs to free its resources, the system can deadlock. The other obvious answer is to reject: the target refuses requests it cannot take and the requester tries again. That avoids holding while waiting, but nothing guarantees the retry ever succeeds.
Strategy | How | Strength | Weakness |
|---|---|---|---|
Pre-allocation | The requester reserves the target resource before sending (credits for target trackers, buffer space for the response) | No request ever waits in the fabric; no retries | The requester must hold credits for every target it might use: expensive with many targets |
Retry (reject) | The target accepts if it can, otherwise rejects; the requester resends later | No credits needed in advance; scales to many requesters and targets | Wasted attempts; no guarantee of progress (livelock, lesson 9.8) |
Retry plus protocol credit | A rejected requester is remembered; when space frees, the target grants a credit to it; the retry with that credit is guaranteed to be accepted | No holding while waiting, no endless retries, no credits needed in advance | The target keeps a list of who it owes a credit |
Link-level credits (Part 4) guarantee a request has room in the next buffer on the link. They say nothing about whether the destination has a tracker for it. Pre-allocating destination resources from every requester to every target would need credits for every pair, which does not scale. Retry with a protocol credit gets the guarantee only when it is needed: after a rejection.
Protocol | Approach |
|---|---|
AMBA CHI | A home node without tracker space answers a request with a retry (RetryAck) and later sends a protocol credit (PCrdGrant); the retried request, carrying that credit, is guaranteed to be accepted. Link credits separately guarantee space on each link. |
PCI Express | Pre-allocation through credits for each traffic type at each link (Part 4), plus endpoints reserving space for completions before issuing reads (lesson 9.5). |
Original AHB | Plain retry (RETRY and SPLIT responses) without a credit guarantee; AHB-Lite dropped them (lesson 9.8). |
A fabric (illustrative) used plain retry at a busy home node. Under contention, some requesters were rejected thousands of times in a row because others' retries happened to arrive just as each tracker freed. Adding a list of rejected requesters and reserving freed trackers for them in order, a protocol credit, bounded every requester's wait to one rejection plus its turn in the list.
Waiting requests hold link buffers (hold and wait); if the traffic they block includes what the target needs to free its resources, a cycle forms.
Once: after the rejection, the target reserves a resource for that requester and grants a credit, and the retry with the credit is accepted.
Waiting in the fabric holds resources while waiting; pre-allocation reserves first; retry rejects and resends.
Pre-allocation does not scale to many targets; plain retry can livelock.
Retry plus a protocol credit guarantees acceptance after at most one rejection, without holding or pre-allocating.