A link replays a packet after a CRC error and raises an interrupt to report it. The link sees one such error a minute, and the operating system handles a minute's worth of interrupts that require no action. On another link, errors rise from one a minute to a thousand a second, a sign that a connector is failing, and nobody notices, because each one was quietly corrected. One design reports too much; the other too little.
How should hardware classify and report errors so software sees what matters?
The obvious policy is to report every error the same way, say with an interrupt. Corrected errors then flood software with events that need no action, while the one fatal error looks no different from a thousand harmless ones. The opposite policy, reporting only failures, hides the trend that predicts failure. Errors need classes that say what they left behind, and counting that turns harmless events into a useful signal.
Class | What it left behind | Hardware does | Software does |
|---|---|---|---|
Correctable | Nothing: the error was fixed (replay, ECC correction) | Count it; log the first and summary; interrupt only above a threshold | Watch the rate; act on trends (replace a part, lower a speed) |
Uncorrectable, non-fatal | One transaction is bad; the link or device still works | Log the details (address, request header); report; poison or error-respond | Recover that transaction: retry, end the process, reset the function |
Uncorrectable, fatal | The link or device can no longer be trusted | Log; report urgently; contain (stop traffic) | Reset the link or device, or the system |
The first error in detail: the error type, the request header or address, the time; later errors often follow from the first, so the first is the one to diagnose.
Counters per class and per source, readable by software, so rates and trends are visible without interrupts.
Sticky status bits that survive a reset where possible, so a crash leaves evidence for the next boot.
Protocol | Error classes and reporting |
|---|---|
PCI Express | Correctable errors (for example a link CRC error fixed by replay) and uncorrectable errors, split into non-fatal (that transaction is unreliable, the link works) and fatal (the link is unreliable). With Advanced Error Reporting, devices log errors with details and send error messages to the root port, which logs them; a poisoned TLP is reported as non-fatal. |
Memory systems | Corrected ECC errors are counted and logged; uncorrectable ones are poisoned or raised as machine-check errors. |
The second link in the opening story (illustrative) corrected every error silently, with no counter software could read. When its error rate climbed for weeks before a connector failed outright, nothing in the logs showed it. Adding a readable correctable-error counter, with an interrupt above a threshold, turned the next failing connector into a planned replacement.
Correctable: the replay fixed it and nothing was lost.
Later errors are often consequences of the first; the first is usually the one that explains the failure.
Classify errors by what they leave behind: correctable, uncorrectable non-fatal, uncorrectable fatal.
Count correctable errors and interrupt only on trends; log uncorrectable ones in detail and report.
Keep the first error in detail and counters readable by software; rising corrected rates predict failure.