Yesterday the build was green. Today it is red, and your working tree is clean: you changed nothing. The failure names a tool, not your code. A warning that stayed silent yesterday is an error today, or a report counts one more cell than it did.
The build passed yesterday and fails today and you changed nothing. What moved?
A build result comes from two inputs: your register-transfer level (RTL) source and the tools that read it. RTL describes hardware as registers and the logic between them. Change either input and the result can change. A new simulator release can promote a warning, tighten a check, or fix a quirk your testbench was leaning on. None of that means your design changed.
Continuous integration (CI) makes this concrete. CI means a fresh machine that checks out your code and runs the build on every change. That machine never carries your laptop's history: it installs whatever the install step asks for today. When the install step says the latest, today and yesterday are different tools.
Not every pin holds equally well. The figure below orders four methods from weakest at the top to strongest at the bottom. Read it top down: each rung removes one more way the tool can move under you.
Record versions in the repo. Write the exact versions behind a green run into the project, in tools/versions.md. This detects drift after the fact. It does not prevent it.
Ask for an exact package version. Request the release you tested, not whatever is newest. A fresh machine then installs the same release, while the repository around it can still move.
Use a container image by digest. Name the whole machine image by its content hash instead of its label. The operating system and the toolchain arrive identical on every machine.
Pin each action to a full commit SHA. Name each CI step by the hash of the exact commit it should run. The automation itself cannot move at all.
Notice the direction. Each rung freezes a wider scope: first your memory, then one package, then the whole machine image, then the automation. Stronger rungs cost more upkeep, so teams mix them: exact versions for daily work, digests and full hashes for releases.
This lesson shows no digest and no commit hash, because none was verified for this project and this lesson never invents one. A real image digest looks like the word sha256 followed by a long hexadecimal string, and a real commit hash looks like a long hexadecimal string you copy from the commit itself. Copy both from the tool. Never type them from memory.
Two things still move. First, the runner image label: a label such as ubuntu-24.04 can receive updates, so two runs on the same label can carry different system packages. Second, the package repositories themselves: asking for a package by name alone installs whatever the repository holds today, and old releases leave it over time. Exact versions narrow this gap. Only a digest closes it.
Pinning is risk management, not magic. Record what you can, freeze what matters, and keep the version report in the log so the next failure starts with a comparison instead of a guess.
The project keeps its memory in tools/versions.md, shown below. The table names each tool on each tested platform, with the date of the run and the result that run produced.
# Tool versions this project was tested with
Result of `make check-tools && make all` plus `python3 model/mini_controller_model.py`, run 2026-10-21.
| Tool | macOS (Homebrew) | Ubuntu 24.04.5 (apt, aarch64 container) |
|---|---|---|
| Icarus Verilog | 13.0 | 12.0 |
| Verilator | 5.048 | 5.020 |
| Yosys | 0.69+post | 0.33 |
| GNU Make | 3.81 | 4.3 |
| Git | 2.54.0 | 2.43.0 |
| Python | 3 | 3 |
| Result | `make all` passes | `make all` passes |
Not yet tested: Ubuntu on x86-64 (the GitHub runner architecture), Windows with WSL, the CI workflow itself.This file is the project's memory of what green looked like. The numbers in it came from one command, make check-tools, which prints one line per tool. Here is that command on the first machine, measured on macOS with Icarus 13.0, Verilator 5.048, Yosys 0.69+post, Make 3.81 and Git 2.54.0:
# captured terminal output
$ bash scripts/check_tools.sh
iverilog Icarus Verilog version 13.0 (stable) (v13_0)
vvp Icarus Verilog runtime version 13.0 (stable) (v13_0)
verilator Verilator 5.048 2026-04-26 rev vUNKNOWN-built20260426
yosys Yosys 0.69+post (git sha1 143eb14f9cc55d6f8927e68523b0c9d2166ed02c, Release, AppleClang clang++ 21.0.0.21000101)
make GNU Make 3.81
git git version 2.54.0 (Apple Git-157)And here is the same command on the second machine, measured on Ubuntu 24.04 with Icarus 12.0, Verilator 5.020, Yosys 0.33, Make 4.3 and Git 2.43.0. The packages are older, and make all passes on both machines, measured on each platform with the versions named above:
# captured terminal output
$ bash scripts/check_tools.sh
iverilog Icarus Verilog version 12.0 (stable) ()
vvp Icarus Verilog runtime version 12.0 (stable) ()
verilator Verilator 5.020 2024-01-01 rev (Debian 5.020-1)
yosys Yosys 0.33 (git sha1 2584903a060)
make GNU Make 4.3
git git version 2.43.0Both version reports above are captured terminal output. The macOS report and the Ubuntu report come from real runs of the project's check script, and make all passes on both platforms with the versions each report names.
Keep this file alive with a habit: after a green run, run make check-tools, paste the output into tools/versions.md with the date and the platform, and commit it. The next red build then starts with a diff of this file.
No. The tool moved, not your code. The versions file tells you exactly what changed, so you compare the two reports first and pin the tested release instead of debugging RTL that never moved.
Run the same check with no tools installed, and watch it say so plainly instead of failing later with a strange message:
# captured terminal output
$ PATH=/usr/bin:/bin bash scripts/check_tools.sh
iverilog MISSING
vvp MISSING
verilator MISSING
yosys MISSING
make GNU Make 3.81
git git version 2.54.0 (Apple Git-157)
[exit status 1]Every tool reports MISSING, and the script exits with status 1, which fails the step at once. The fix is mechanical: install the tools, re-run make check-tools until every line names a version, and only then run the build.
Pinning only some tools. You freeze the simulator and forget the synthesis tool, so the simulation lane reproduces and the synthesis lane drifts. Pin every tool the build reads, or say which ones you left out.
Trusting a tag that can move. A runner label or an action reference such as checkout@v4 can point at new content tomorrow. This project's own workflow file carries a note to pin its action to a verified commit hash. Read tags as promises, and replace the load-bearing ones with hashes.
Recording on one machine and testing on another. A versions file written on macOS says nothing about the CI machine. Record the report on every platform you build on, and compare each failure against its own platform's file.
Rule: when two runs must agree, freeze everything both runs read: the code, the tools, and the steps.
The toolchain moved, not your code. Diff the two version reports to name the changed tool, pin the tested release in the install step, and re-run. Debugging the RTL first would chase a change that never happened.
The first two: record versions in the repo and ask for exact package versions. Both are cheap and catch most drift. Save digests and commit hashes for release builds, where the upkeep pays off.
Run make check-tools in the project, paste the output into tools/versions.md with today's date and your platform, and commit it. A future red build then starts with a diff instead of a guess.
A build result has two inputs: your source and your tools.
Pin in layers: record versions, ask exact versions, freeze images by digest and actions by hash.
Keep the version report in every log, so the next red build starts with a comparison.