On a 4-core chip, every cache miss is shown to every other cache: three lookups per miss, cheap and fast. The same design on 64 cores shows every miss to 63 caches. Most of them have no copy and answer "not here". Snoop traffic grows with the number of cores times the miss rate, and at some point the caches spend more time answering questions than serving their own core.
How does a coherence protocol find the other copies of a line, and how does that scale?
The obvious answer is to ask everyone: broadcast the request, and whoever has a copy acts. That is snooping, and on a small system with a shared bus it is ideal: the bus orders all requests, and every cache sees them in the same order. But every request costs a lookup in every cache and a message on every link, needed or not. Bandwidth and lookup power grow with the number of caches, while most caches, for most lines, have nothing to say.
Snooping | Directory | |
|---|---|---|
Finding copies | Broadcast every request to all caches | Look up the line's entry at its home node: which caches hold it, in what state |
Messages per request | One to every cache | Only to the caches that hold it (often none) |
Ordering | A shared bus or ordered broadcast network orders requests | The home node orders requests for its lines |
Latency | Direct: requester to all, owner replies | Indirect: requester to home, home to owner, owner to requester (lesson 13.5) |
Storage | None beyond the caches | A directory entry per tracked line (sharer bits or pointers) |
Scales to | Small core counts | Many cores |
A snoop filter records which lines are present in which caches (or at least which are certainly absent), and drops snoops that would find nothing.
It keeps snooping's simplicity for small groups while cutting most of the useless traffic.
Inclusive directories and filters must track every cached line; when an entry is evicted, the caches holding that line must be invalidated ("back-invalidation"), which costs performance if the filter is too small.
Protocol | Finding copies |
|---|---|
AMBA ACE | Snooping: the interconnect sends snoop requests on the AC channel to masters' caches, which reply on CR and CD (fact A23); interconnects commonly add snoop filters. |
AMBA CHI | A home node per address range orders requests and sends snoops; designed for large, directory-style systems (fact A23). |
Large multi-socket systems | Directories or directory caches at the memory's home, to avoid broadcasting across sockets. |
A many-core design (illustrative) used a snoop filter sized for the typical working set. Under one database workload, the filter thrashed: each eviction back-invalidated lines still in use, and those cores then missed on lines they had just been using. Throughput fell by a third. Enlarging the filter, and making it track lines per cluster rather than per core, fixed it.
None: the directory shows no holders, so memory supplies the line directly.
The bus (every cache sees requests in the same order); in a directory system, the line's home node.
Snooping broadcasts every request; it is simple and fast but its traffic grows with the number of caches.
A directory records who holds each line and messages only them, at the cost of an extra hop and storage.
Snoop filters cut useless snoops; size them for real working sets to avoid back-invalidation.