A flat list of violations hides what happened since yesterday. Classifying each endpoint by its history tells you what to discuss: new violations point at recent changes, regressions at fixes that broke something, stuck endpoints at fixes that do not work.
TOL = 0.002 # consecutive days with |change| below this count as unchanged
# endpoint: (slack yesterday or None if not failing yesterday, slack today or None if absent today, days unchanged)
today = {
"e1": (None, -0.020, 0), # new
"e2": (-0.030, -0.045, 0), # regressed: worse than yesterday by more than TOL
"e3": (-0.015, -0.016, 8), # stuck: eight consecutive days within TOL
"e4": (-0.060, -0.020, 0), # improving
"e5": (0.010, -0.004, 0), # new: was passing yesterday
"e6": (-0.040, None, 0), # dropped: absent from today's run
}
def classify(prev, cur, days_unchanged):
if cur is None:
return "dropped"
if prev is None or prev >= 0:
return "new"
if cur < prev - TOL:
return "regressed"
if abs(cur - prev) <= TOL:
return "stuck" if days_unchanged >= 5 else "unchanged"
return "improving"
cls = {e: classify(*v) for e, v in today.items()}
assert cls == {"e1": "new", "e2": "regressed", "e3": "stuck", "e4": "improving", "e5": "new", "e6": "dropped"}
counts = {k: sum(1 for v in cls.values() if v == k) for k in ("new", "regressed", "stuck", "unchanged", "improving", "dropped")}
assert counts == {"new": 2, "regressed": 1, "stuck": 1, "unchanged": 0, "improving": 1, "dropped": 1}
print("PASS: triage buckets", counts)Bucket | Endpoints | Question for the meeting |
|---|---|---|
New | e1, e5 | What changed yesterday on these paths? |
Regressed | e2 | Which fix made it worse? |
Stuck | e3 | Why has the same fix not worked? Is a different kind needed? |
Improving | e4 | On track; no discussion needed |
Dropped | e6 | Absent today: fixed, renamed or lost from the run? |
Two endpoints are new: e1 was never failing before, and e5 was passing yesterday at +0.010 and fails by 4 ps today. One regressed (e2 went from -0.030 to -0.045, well beyond the 2 ps tolerance), one is stuck after eight consecutive days within the tolerance, and one is improving. A sixth endpoint, e6, is absent from today's run: it is counted as dropped, not as fixed, until someone confirms why it disappeared. The meeting spends its time on the new, regressed, stuck and dropped buckets and skips the improving one. A tolerance of 0.002 ns and a classification threshold of five unchanged days for 'stuck' are choices; use what your cadence needs.
Triage is computed from the two most recent runs, using the same scenarios and endpoint sets, not by eye.
Regressions are traced to the ECO or constraint change that caused them, the same day.
Stuck endpoints get a different approach assigned, not another round of the same one.
Unchanged, not yet stuck: the 1 ps move is within the 0.002 ns tolerance but it has only two unchanged days, and stuck needs five. It still deserves a look if a fix applied yesterday was meant to help.