Spend per task misleads, because rejected outputs still cost money. The decision figure is spend divided by accepted results, which charges the misses to the wins.
The example below reuses synthetic fixture figures with a stated example spend. All spend numbers here are example parameters in abstract units, not prices of any real service. This is tool spend per accepted result, not total engineering cost: record review, independent verification, repair, and baseline effort separately. With zero accepted results the ratio is undefined, not zero.
assume tool spend for the run = 70 units (stated example parameter)
accepted results = 14
cost per accepted result = 70 / 14 = 5.0 units
per-task spend = 70 / 20 = 3.5 units, but the accepted-result figure guides the decision
shares: accepted 14 / 20 = 70%, rejected 6 / 20 = 30%; 70% + 30% = 100%Compare at most two options plus the baseline on the same frozen set. More options at once invite tuning to the set instead of reading it.
Option | Pass rate on 20 tasks | Time kept | Example spend per accepted result |
|---|---|---|---|
Baseline (no tool) | Must be independently graded; not perfect by definition | Reference time; 770 min in the invented fixture | No candidate-tool spend; human effort still counts |
Setup A (synthetic fixture) | 14 of 20; illustrative Wilson interval 48.1% to 85.5% | Hypothetical 315 min of 770; equivalent quality not established | 5.0 fictional units |
Setup B (example) | Needs its own frozen run | Needs its own timed run | Spend divided by its own accepted count |
Adopt only with a written threshold met: a minimum pass rate, a minimum kept time, and a maximum cost per accepted result. Behaviour varies by version, so the decision names the exact setup and date.
Schedule the re-check before adopting, and rerun the frozen set when the tool, the task mix, or the prices behind the units change. A passed evaluation expires with its conditions.
7.0 units. Each accepted result carries one rejected one, so 3.5 divided by 0.5 is 7.0.
No, not without a rerun. Behaviour varies by version, so the frozen set must be rerun against the new setup before claiming anything.