A network accelerator is being powered down to save energy. Software stops sending it work, waits a millisecond to be safe, and switches it off. Most of the time that works. Occasionally, the accelerator had a write still sitting in its output buffer, or a read still waiting for data from memory, and the switch-off loses it: a corrupted packet in one case, a hung read in the other. Waiting "long enough" is not a protocol.
How is an interface brought to a clean stop before its power is removed or it is reset?
The obvious answer is to stop sending new work and wait a while. There is no time that is always long enough: a request can be stuck behind congestion or a slow target for an unbounded time. And stopping new work is only one direction; responses still owed, writes still buffered and credits still held all need to finish too. Quiescing must be driven by evidence, not by a timer.
Step | Why it matters |
|---|---|
Stop new requests | Otherwise the drain never ends; new requests should be held upstream or answered with errors, not dropped |
Complete outstanding requests | Responses arriving after power-down are lost; requests left at the target hang their requesters |
Flush buffers | Posted writes in a buffer look complete to the sender but have not reached their target |
Settle credits | Credits held by the departing side must be accounted for, or the partner's count is wrong after the block returns |
Isolate, then cut | Outputs of a powering-down block float; isolation cells clamp them to safe values first |
Count outstanding requests: increment on issue, decrement on completion; zero means drained in that direction.
Use the protocol's completion signals: write responses, completions, acknowledgements; a write is done when its response returns, not when it leaves.
Fence or barrier: issue a barrier and wait for it to complete; ordering rules then guarantee everything before it is done (lesson 7.5).
Bound it: a request that never completes must still end, by a timeout that escalates to an error or a reset (lesson 9.9).
Protocol | Quiescing |
|---|---|
PCI Express | The Transactions Pending bit in Device Status shows non-posted requests still awaiting completions; software waits for it to clear (with a timeout) before a Function Level Reset or power-down (fact P36). |
AMBA Q-Channel | The device accepts a quiescence request only when it is idle, and QACTIVE shows whether it still has activity (fact A22). |
DCLink | The device accepts |
The opening accelerator (illustrative) was fixed by replacing the millisecond wait with a drain sequence: block new work, poll an outstanding-request counter until zero, issue a write barrier and wait for its response, then accept the power-down. The rare corrupted packets and hung reads disappeared, and power-down got faster, since most drains completed in microseconds.
A posted write gets no response, so it looks finished to the sender while it may still sit in a buffer on the way to its target.
It should be held upstream until the block returns, or answered with an error; never silently dropped.
Quiescing is driven by evidence: stop new work, complete every outstanding request, flush, settle credits.
Only after confirming idle may the block accept power-down; outputs are isolated before power goes.
Bound the drain with a timeout that escalates, and make each step observable.