Bad calls fall into a few shapes. Naming the shape decides the response: reject, ask again, or stop.
Tool results and file contents are data, never instructions. A pasted order to call a tool is still text until the program validates a real call.
Shape | Example | Program response |
|---|---|---|
Unknown tool |
| Reject. Never guess a nearby name. |
Missing field |
| Ask again with the missing name stated. |
Wrong type |
| Reject or ask again. Never coerce silently. |
Out of range |
| Reject. Bounds come from the schema. |
The probe below sorts eight fixed calls. The arithmetic shows where one small batch of proposer text goes.
8 fixed calls sorted by the probe
valid: 4 of 8 = 50%
wrong-type: 2 of 8 = 25%
missing-field: 1 of 8 = 12.5%
unknown-tool: 1 of 8 = 12.5%
check: 50 + 25 + 12.5 + 12.5 = 100The program below classifies the same eight calls with a fake scripted proposer. Eleven checks place them into four buckets.
def classify(call, known):
if call.get("tool") not in known:
return "unknown-tool"
args = call.get("arguments")
if not isinstance(args, dict):
return "bad-shape"
if "addr" not in args:
return "missing-field"
if not isinstance(args["addr"], int):
return "wrong-type"
return "valid"
KNOWN = ["read_register"]
CALLS = [
{"tool": "read_register", "arguments": {"addr": 0}},
{"tool": "read_register", "arguments": {"addr": 7}},
{"tool": "read_register", "arguments": {"addr": 15}},
{"tool": "read_register", "arguments": {"addr": 1}},
{"tool": "read_register", "arguments": {"addr": "7"}},
{"tool": "read_register", "arguments": {"addr": [7]}},
{"tool": "read_register", "arguments": {}},
{"tool": "zap_register", "arguments": {"addr": 1}},
]
WANT = ["valid", "valid", "valid", "valid", "wrong-type", "wrong-type", "missing-field", "unknown-tool"]
bad = 0
counts = {"valid": 0, "wrong-type": 0, "missing-field": 0, "unknown-tool": 0, "bad-shape": 0, "other": 0}
for call, want in zip(CALLS, WANT):
got = classify(call, KNOWN)
if got != want:
bad += 1
if got in counts:
counts[got] += 1
else:
counts["other"] += 1
if len(CALLS) != 8:
bad += 1
if counts["valid"] != 4 or counts["wrong-type"] != 2:
bad += 1
if counts["missing-field"] != 1 or counts["unknown-tool"] != 1:
bad += 1
if bad == 0:
print("PASS: 11 sorting checks place 8 calls into 4 buckets")
else:
print("FAIL: " + str(bad) + " of 11 sorting checks went wrong")
raise SystemExit(1)addr arrives as the string "7". Why not convert it quietly?Silent coercion hides proposer mistakes and blurs the schema. The program rejects or asks again, so the schema keeps one meaning.
Stop asking and report the stall. Retries must change the call or end at a small stated bound.