The program below stands in for a model with a fake scripted function. It fixes the first open issue each round and plants a bad edit on two rounds.
Part | Role in the program |
|---|---|
fake_lint | Counts open issues in the scratch state. |
fake_tests | Fails when a bad edit is present. |
fake_fixer | Scripted stand-in: fixes one issue, plants a bad edit on rounds 1 and 2. |
Acceptance rule | Keeps a patch only on fewer hits plus green tests. |
Stop rule | Stops at 5 rounds, at zero hits with green tests, or after 2 stalled rounds. |
The run starts from 3 open issues and ends with 2 left for a human. Every number in the next section follows from these bounds.
import sys
MAX_ROUNDS = 5
STALL_LIMIT = 2
state = ["A", "B", "C"]
rounds = 0
accepted = 0
stalled = 0
history = [len(state)]
def fake_lint(items):
return list(items)
def fake_tests(items):
return "BAD" not in items
def fake_fixer(open_issues, round_no):
# FAKE scripted stand-in for a model: fixes the first open issue,
# and plants a bad edit on rounds 1 and 2.
if len(open_issues) == 0:
return []
if round_no == 1 or round_no == 2:
return ["fix:" + open_issues[0], "BAD"]
return ["fix:" + open_issues[0]]
def apply_patch(items, patch):
out = list(items)
for token in patch:
if token.startswith("fix:"):
name = token[4:]
if name in out:
out.remove(name)
elif token not in out:
out.append(token)
return out
while rounds < MAX_ROUNDS:
open_issues = fake_lint(state)
if len(open_issues) == 0 and fake_tests(state):
break
patch = fake_fixer(open_issues, rounds)
candidate = apply_patch(state, patch)
if len(fake_lint(candidate)) < len(fake_lint(state)) and fake_tests(candidate):
state = candidate
accepted += 1
stalled = 0
else:
stalled += 1
rounds += 1
history.append(len(fake_lint(state)))
if stalled >= STALL_LIMIT:
break
if rounds != 3:
print("FAIL: rounds is " + str(rounds) + ", wanted 3")
sys.exit(1)
if accepted != 1:
print("FAIL: accepted is " + str(accepted) + ", wanted 1")
sys.exit(1)
if stalled != 2:
print("FAIL: stalled is " + str(stalled) + ", wanted 2")
sys.exit(1)
if state != ["B", "C"]:
print("FAIL: leftover state is " + str(state))
sys.exit(1)
if history != [3, 2, 2, 2]:
print("FAIL: history is " + str(history))
sys.exit(1)
print("PASS: 3 rounds, 1 patch kept, 2 stalled rounds, 2 issues left for a human")Round 0 fixes issue A and the patch is kept, so the count falls from 3 to 2. Rounds 1 and 2 each pair a fix with a bad edit, so the count stays at 2 and both patches are dropped.
Two consecutive stalled rounds hit the stall limit, so the loop stops after 3 rounds. One patch is kept, 2 issues remain, and the history reads 3, 2, 2, 2.
It also plants a bad edit, so the count does not fall and the suite turns red. Both halves of the acceptance rule fail.
The stall limit. Only 3 of the 5 allowed rounds run before 2 consecutive stalled rounds stop the loop.