Not if only data is lost. The detecting unit often has no idea who will use the line, and may not be using it itself: a cache answering a forward, a memory controller filling a request, a scrubber. Stopping at detection punishes whoever happened to touch the line. The alternative is poison: mark the data as known-bad, keep moving it through the protocol like any other data, and raise an error only when some core or device actually consumes it.
Rule | Mechanism |
|---|---|
Poison travels with the data | Every Data message carries a poison bit beside the payload. Buffers at H keep it. Memory stores it, for example as a reserved ECC pattern. |
Coherence runs normally | States, directory updates, Inv, acks and Put-Ack are unchanged. The line still has one legal state per cache. |
Consumption raises the error | A load that returns poisoned data to a core (or a device read of it) reports a synchronous error to that consumer instead of returning a value. |
Nothing else does | Moving, forwarding, writing back or evicting a poisoned line raises no error. The detection is logged, as a report, not an interruption. |
Only new data clears it | A partial store merges into the line and keeps the poison. Writing every byte of the poison granule replaces the bad data and clears it. |
In this model the poison granule is the whole 64-byte line, so X is poisoned even if the bad bits are elsewhere in the line. Designs may track poison per smaller chunk.
MSI, extending the dirty read of MSI Protocol with its roles: from X = 0, C0 acquired M and stored 1, so C0 M / 1, H exclusive(C0), memory = 0. C1 sends GetS. When the Fwd-GetS arrives, C0 reads the line out of its data array and its SECDED check reports a double flip: uncorrectable. The value 1 is gone. C0 does the MSI downgrade as usual but sends Data with the poison bit set, written Data(poisoned). The messages are those of the linked trace, so only the table is shown, with a poison column.
Step | C0 state / value | C1 state / value | H directory | Memory | Newest data | Poisoned copies |
|---|---|---|---|---|---|---|
0 warm-up | M / 1 | I / - | exclusive(C0) | 0 | C0: 1 | none |
1. GetS from C1 at H | M / 1 | IS^D / - | exclusive(C0), pend GetS C1 | 0 | C0: 1 | none |
2. Fwd-GetS at C0, ECC finds a double flip | S / poisoned | IS^D / - | exclusive(C0), pend GetS C1 | 0 | none: 1 is lost | C0, Data(poisoned) in flight to H |
3. Data(poisoned) at H | S / poisoned | IS^D / - | exclusive(C0), pend GetS C1 | 0 | none: 1 is lost | C0, H buffer |
H writes memory with poison | S / poisoned | IS^D / - | S{C0}, pend GetS C1 | poisoned | none: 1 is lost | C0, memory, Data(poisoned) in flight to C1 |
4. Data(poisoned) at C1 | S / poisoned | S / poisoned | S{C0}, pend GetS C1 | poisoned | none: 1 is lost | C0, C1, memory |
(assumed) grant delivered | S / poisoned | S / poisoned | S{C0,C1} | poisoned | none: 1 is lost | C0, C1, memory |
L. C1 loads X | S / poisoned | S / poisoned | S{C0,C1} | poisoned | none: 1 is lost | C0, C1, memory, error reported to C1's load |
Read the states column by column: every row is a legal MSI snapshot. There is never a writer beside a reader, and H's entry matches the holders. SWMR holds. What has failed is the data-value invariant: no copy anywhere holds 1. Poison does not repair that; it makes the failure visible to exactly the reader who would otherwise have used a wrong value, and to nobody else.
Three things did not happen. C0 raised no error at step 2, because it was only answering a forward. H raised none at step 3 or when writing memory. And had C1 never loaded X, no error would have been raised at all, beyond a logged report. Without the poison bit, H would have written the garbage bits to memory with fresh, valid ECC, and C1's load would have returned a wrong number with no warning: silent data corruption, the worst outcome in RAS terms.
import random
GARBAGE = {"X": 6, "Y": 3} # what C0's double flip leaves in X's line
def run(seed, variant=None):
rng = random.Random(seed)
c = {0: {"st": "M", "v": {"X": 1, "Y": 5}, "p": False, "flip": True}, 1: {"st": "I", "v": None, "p": False, "flip": False}}
mem = {"v": {"X": 0, "Y": 0}, "p": False}
good = {"X": 1, "Y": 5} # what a correct load must return, None once lost
st = {"stage": None, "errors": 0, "poisoned_loads": 0, "ops": []}
def data(rec): # every Data message carries the poison bit
return {"v": dict(rec["v"]), "p": rec["p"] and variant != "drop_on_data"}
def read_array(k): # ECC check when a line is read out
if c[k]["flip"]:
c[k].update(v=dict(GARBAGE), p=True, flip=False)
good.update(X=None, Y=None) # the only newest copy is gone
st["errors"] += variant == "raise_at_detection"
for _ in range(rng.randint(4, 8)):
ops = [(st["stage"], st.get("req"))] if st["stage"] else []
for k in (0, 1) if not ops else ():
ops.append(("load", k) if c[k]["st"] != "I" else ("fwd_gets", k) if c[1 - k]["st"] == "M" else ("read_mem", k))
ops += [("partial", k), ("full", k)] + ([("puts", k)] if c[k]["st"] == "S" else [])
op, k = rng.choice(ops)
o = 1 - k
st["ops"].append(op)
if op == "fwd_gets": # Fwd-GetS at the M holder: downgrade, Data to H
read_array(o)
c[o]["st"] = "S"
st.update(msg=data(c[o]), req=k, stage="h_buffer")
elif op == "h_buffer":
st.update(buf=st["msg"], stage="h_write_mem") # the buffer keeps the bit
elif op == "h_write_mem":
mem.update(v=dict(st["buf"]["v"]), p=st["buf"]["p"])
st["stage"] = "data_to_req"
elif op == "data_to_req":
c[k].update(st="S", **data(st["buf"]))
st["stage"] = None
elif op == "read_mem": # GetS served from memory
c[k].update(st="S", **data(mem))
elif op == "puts": # clean eviction, PutS carries no data
c[k].update(st="I", v=None, p=False)
elif op == "load":
read_array(k)
w = rng.choice("XY")
if c[k]["p"]: # consumption: the error goes to this load only
assert None in good.values(), "poison left on a fully overwritten line"
st.update(errors=st["errors"] + 1, poisoned_loads=st["poisoned_loads"] + 1)
else:
assert c[k]["v"][w] == good[w], "wrong value returned silently"
else: # partial or full store: GetM (Data from the M holder or memory), then write
if c[k]["st"] == "I":
read_array(o) if c[o]["st"] == "M" else None
line = data(c[o] if c[o]["st"] == "M" else mem)
else:
read_array(k)
line = {"v": dict(c[k]["v"]), "p": c[k]["p"]}
c[o].update(st="I", v=None, p=False)
n = rng.randrange(10, 99)
if op == "partial": # merge into X only, the poison stays
line["v"]["X"], good["X"] = n, n
line["p"] = line["p"] and variant != "clear_on_partial"
else: # every byte of the granule replaced
line, good = {"v": {"X": n, "Y": n + 1}, "p": False}, {"X": n, "Y": n + 1}
c[k].update(st="M", **line)
held = [r for r in c.values() if r["st"] != "I"]
held += [mem] if not st["stage"] and all(r["st"] == "S" for r in held) else [] # memory current
assert all(r["p"] or r["v"] == good for r in held), "lost or stale bits not marked poisoned"
assert not (any(r["st"] == "M" for r in c.values()) and "I" not in (c[0]["st"], c[1]["st"])) # SWMR
assert st["errors"] == st["poisoned_loads"], "error raised without a poisoned load"
return st
stats = [run(seed) for seed in range(400)]
assert sum(s["poisoned_loads"] for s in stats) > 0 and sum(s["ops"].count("full") for s in stats) > 0
assert {"fwd_gets", "h_buffer", "h_write_mem", "data_to_req", "read_mem", "puts", "load", "partial", "full"} <= {o for s in stats for o in s["ops"]}
for variant in ("drop_on_data", "raise_at_detection", "clear_on_partial"):
failures = set()
for seed in range(400):
try:
run(seed, variant)
except AssertionError as e:
failures.add(str(e))
assert failures, variant # every broken variant is caught
print("PASS: errors arise only at poisoned loads, nothing wrong is silent, and all three broken variants fail")The check models X's line as a record of values and a poison bit, starting from the warm-up above with C0's double flip not yet read out. With a fixed seed it runs 400 random sequences of 4 to 8 legal operations: the Fwd-GetS at C0, H buffering the Data, H writing memory, the Data to C1, a PutS from either cache, a re-read from memory, loads at C0 or C1, a partial store to X and a full overwrite of the line. After every step it checks that each copy either matches the true value or is marked poisoned, that every error belongs to a load of poisoned data, and that SWMR holds. A partial store must keep the poison and a full overwrite must clear it. Three broken designs each fail: a Data path that drops the bit (C1 holds garbage marked good), an error raised at detection (an error with no consumer) and a partial store that clears the poison (the other bytes are still garbage).
Poison is a mark on a payload: it says 'these bytes are bad'. It needs a correct address, a correct state and a correct directory entry around it, so that the poisoned line still moves through a legal protocol. If the uncorrectable error is in a state bit, a tag or a directory entry, there is no payload to mark. H does not know who may write or who must be invalidated, and a cache with a bad tag does not know which address its line belongs to. Marking the line poisoned would not make the next grant safe. Metadata errors need a different answer: rebuild the metadata, as in the next lesson, or stop.
Yes, in this model. GetM proceeds normally (C1 goes to SM^D, H sends Inv to C0, collects its Inv-Ack and grants M with permission-only Data, since C1 already holds the line), and C1's store of X writes one location inside the 64-byte granule. The other bytes are still bad, so the merged line stays poisoned. Only a write of every byte of the granule clears the poison.