A parallel analysis adds up many slack numbers in an order that can change from run to run. Floating-point addition is not associative, so totals such as TNS can differ in their last digits even though nothing is wrong. A tolerance policy separates these harmless differences from real ones.
import math
vals = [0.1, 0.2, 0.3]
forward = 0.0
for v in vals: # (0.1 + 0.2) + 0.3, added one at a time
forward += v
backward = 0.0
for v in reversed(vals): # (0.3 + 0.2) + 0.1
backward += v
exact = math.fsum(vals) # correctly rounded sum
assert forward != backward
assert abs(forward - backward) < 1e-15 and abs(forward - exact) < 1e-15
# a tolerance policy: differences below 0.1 ps (1e-4 ns) are noise; above are real
tol = 1e-4
runs_a = [-0.0205, 0.0153, 0.0402]
runs_b = [-0.02050000000000001, 0.0153, 0.0402]
runs_c = [-0.0217, 0.0153, 0.0402]
def same(x, y):
return all(abs(p - q) <= tol for p, q in zip(x, y))
assert same(runs_a, runs_b) and not same(runs_a, runs_c)
print("PASS: forward", forward, "backward", backward, "differ by", abs(forward - backward), "; tolerance", tol, "ns accepts run b and rejects run c")Source of nondeterminism | Effect | Handling |
|---|---|---|
Summation order in parallel code | Last-digit differences in totals | Compare with a tolerance |
Thread scheduling affecting the order of tie-breaking | Different but equally valid choices | Fix seeds or ordering where the tool allows; otherwise accept within the tolerance |
Timestamps and file ordering in scripts | Different logs, rarely different results | Normalise before diffing |
Unrecorded input | Real result differences | Add to the manifest |
The sum of 0.1, 0.2 and 0.3 is 0.6000000000000001 in one order and 0.6 in the other: a difference of about 1e-16. A tolerance of 0.1 ps (1e-4 ns) accepts differences of that size and rejects the 1.2 ps difference of run c, which is real. The tolerance should be well below the margin you care about (a picosecond or less) and well above the floating-point noise; the policy is written down once so nobody argues about it each time.
A documented tolerance, used by every comparison script.
Differences above the tolerance with identical manifests are logged as defects, not ignored.
Seeds and thread settings are in the manifest where they influence results.
No: the 0.1 ps tolerance is per endpoint, so a TNS difference must be compared against 0.1 ps times the number of violating endpoints, or compare per endpoint first. A 2 ps TNS gap across only a handful of endpoints is well above that allowance, so it must be investigated as nondeterminism or an unrecorded input.