Writing Good Properties: Decomposing Intent into Checkable Guarantees: Expanded Case Study

Extended case study

A formal regression involving Writing Good Properties: Decomposing Intent into Checkable Guarantees reopens late in the release cycle after RTL and constraint updates.

Background

Earlier runs were stable, but model assumptions drifted and property intent was not re-audited after implementation changes.

Symptoms observed

• non-vacuous closure rate, counterexample turnaround time, and requirement-level residual risk trend trends worsen while status dashboards look superficially stable.

• counterexample patterns recur across related properties.

• reviewers disagree on whether failures are real bugs or modeling artifacts.

Investigation timeline

1. Hour 0: freeze RTL, assumptions, and tool settings for reproducibility.

2. Hour 1: classify failures into bug, model mismatch, or weak-property buckets.

3. Hour 2: isolate first divergence and map to requirement intent.

4. Hour 3: apply one constrained change and rerun focused property set.

5. Hour 4: confirm reachability and vacuity quality did not regress.

6. Hour 5: replay representative traces in simulation or equivalent flow.

7. Hour 6: publish closure memo with residual risk classification.

Root cause

Root cause traced to Writing Good Properties: Decomposing Intent into Checkable Guarantees: Strong property authoring starts from requirement decomposition: split a broad requirement into trigger, preconditions, transition rule, and completion rule.

Fix and validation

• Correct assumption/property scope to preserve legal behavior.

• Add targeted helper checks that expose key intermediate invariants.

• Update runbook and requirement traceability for future regression stability.

Lessons learned

• Status color is not proof quality; audit supporting evidence.

• First-divergence classification outperforms broad trace inspection.

• Constraint and abstraction governance must be versioned and reviewed.

CASE STUDY - Writing Good Properties: Decomposing Intent into Checkable Guarantees
closure slope / vacuity trend / inconclusive aging / replay confidence

Formal deep dive

Property and constraint engineering is successful when decomposition, reuse, and abstraction preserve legal behavior.

Concept diagram

PROPERTY DEVELOPMENT PIPELINE

spec clause -> decomposed properties -> constraints -> covers -> closure packet

Metric graph

CONSTRAINT HYGIENE TREND

over-constraint risk    ████
cover reachability      ███████
library consistency     █████

Metrics and artifacts to collect

• assume/assert separation coverage

• critical cover reachability score

• checker library adoption and drift

• over-constraint warning trend

Mini case study

A reusable checker library reduced regression noise after assumptions were explicitly documented and reviewed per IP.

Debug branches

• Review every assumption against a spec citation.

• Use covers to confirm legal corner scenarios remain reachable.

• Track abstraction choices in a rollback-ready ledger.

Senior review question

Ask: which requirement intent is proven, under which assumptions, and what residual risk remains?

Key takeaways

• Tie each proof claim to assumption boundaries and reachability evidence.

• Prefer minimal reversible fixes and preserve legal behavior visibility.

Common pitfalls

• Treating runtime reduction as proof-quality improvement without audits.

• Declaring closure while critical covers remain unreachable.

• Using broad waivers instead of first-divergence root-cause ownership.

Principal formal review addendum

Writing Good Properties: Decomposing Intent into Checkable Guarantees should be reviewed as a requirement-evidence workflow, not a single status report.

Use non-vacuous closure rate, counterexample turnaround time, and requirement-level residual risk trend as the monitoring lens and formal closure packet: assumptions audit, proof status matrix, counterexample classification, and requirement traceability as closure proof.

Property development is architecture translation work: requirement intent must survive decomposition, abstraction, and reuse. Strong teams preserve legal reachability while improving convergence.