Typical FPV Tool Flow: Setup, Constraints, Run, Debug, and Closure: Reports and Metrics

Reports and metrics

Reports and Metrics for Typical FPV Tool Flow: Setup, Constraints, Run, Debug, and Closure is anchored on non-vacuous closure rate, counterexample turnaround, and residual-risk trend by requirement class. Convert outcomes into assumption-aware, evidence-backed actions.

A useful report explains why closure quality moved, not only that status changed.

Evidence matrix

EVIDENCE MATRIX - Typical FPV Tool Flow: Setup, Constraints, Run, Debug, and Closure

+-----------------------------+--------------------------------+--------------------------------+---------------------------+
| Evidence                    | Tells you                      | Does not prove                 | Next action               |
+-----------------------------+--------------------------------+--------------------------------+---------------------------+
| property status by class    | closure shape by requirement   | model realism                  | pair with cover reachability |
| vacuity and trigger checks  | assertion meaningfulness       | full legal-path exploration    | inspect assumptions       |
| counterexample traces       | concrete divergence path       | complete bug-space closure     | classify and replay       |
| assumption audit trail      | model boundary confidence      | implementation correctness     | review spec traceability  |
| before/after trend packet   | mitigation movement quality    | long-window stability          | run broader matrix        |
+-----------------------------+--------------------------------+--------------------------------+---------------------------+

• Track non-vacuous closure rate, counterexample turnaround, and residual-risk trend by requirement class by requirement class and risk tier.

• Include assumption and tool metadata in every report header.

• Correlate status with vacuity and cover reachability movement.

• Call out contradictory evidence explicitly.

Formal deep dive

FPV foundations are reliable only when assumptions, reset semantics, and requirement intent are explicitly modeled and audited.

Concept diagram

FPV FOUNDATION LOOP

requirements -> property set -> assumptions and reset model -> prove/fail traces -> closure audit

Metric graph

FOUNDATION HEALTH

vacuous passes         ████
reachable proofs       ███████
inconclusive backlog   █████
reopened properties    ███

Metrics and artifacts to collect

• assumption traceability matrix

• vacuity and reachability status

• proof core relevance summary

• counterexample classification trend

Mini case study

A green-looking run was invalidated after legal-mode covers failed, exposing assumptions that removed realistic traffic.

Debug branches

• Validate requirement-to-property mapping before tuning runtime.

• Check legal scenario reachability after every assumption change.

• Classify first divergence as model issue or RTL bug.

Senior review question

Ask: which requirement intent is proven, under which assumptions, and what residual risk remains?

Key takeaways

• Tie each proof claim to assumption boundaries and reachability evidence.

• Prefer minimal reversible fixes and preserve legal behavior visibility.

Common pitfalls

• Treating runtime reduction as proof-quality improvement without audits.

• Declaring closure while critical covers remain unreachable.

• Using broad waivers instead of first-divergence root-cause ownership.

Report interpretation

Report status with vacuity, cover reachability, and assumption influence side by side.

Promote only when trend data supports reproducible closure behavior.