Change One Thing at a Time · ai-skills

Asking Clearly · Beginner to Intermediate

On this page

One variable per edit

Edits that change several elements at once hide their own cause. When the set improves, no one knows which edit did it.

One element per version keeps the cause visible. The log records the version, the single change, and the set score.

The edit loop

Each version follows the same four steps. The set from the next lesson supplies the score.

Step	Action	Record
Freeze	Copy the current prompt as a numbered version.	Version number and full text.
Change	Alter exactly one element: a fact, a definition, or the shape.	Which element changed and why.
Rerun	Run every test prompt, not just the failing one.	Passing count over total count.
Keep or revert	Keep versions that raise the score without breaking others.	New score beside the old score.

A worked loop

One version pins the output shape and the set moves from 2 of 6 to 3 of 6. Repeated runs then check the gain is stable.

Sampling rescales with temperature, and temperature 0 is approximately greedy. That is not a guarantee of identical output across runs, so rerun even at temperature 0.

plaintext

agreement across 5 runs of one version (example counts):
identical replies: 4 -> 4 / 5 = 80%
set score after the edit: 3 of 6 hold, up from 2 of 6
temperature 0 is approximately greedy, not identical: rerun before judging an edit

Common mistakes

Check yourself Version 5 adds two facts and a format example at once, and the set improves. What can be concluded?

Little. Two edits confound the cause, so each edit needs its own rerun.

Check yourself Five runs of one version at temperature 0 give 4 identical replies. What does that show?

Agreement of 4 of 5, or 80 percent. Temperature 0 is approximately greedy rather than identical, so rerun before judging an edit.

Related topics

Up nextKeep a Small Set of Test Prompts

Practice this lesson Report an error

The edit loop

Each version follows the same four steps. The set from the next lesson supplies the score.

Step	Action	Record
Freeze	Copy the current prompt as a numbered version.	Version number and full text.
Change	Alter exactly one element: a fact, a definition, or the shape.	Which element changed and why.
Rerun	Run every test prompt, not just the failing one.	Passing count over total count.
Keep or revert	Keep versions that raise the score without breaking others.	New score beside the old score.

A worked loop

One version pins the output shape and the set moves from 2 of 6 to 3 of 6. Repeated runs then check the gain is stable.

Sampling rescales with temperature, and temperature 0 is approximately greedy. That is not a guarantee of identical output across runs, so rerun even at temperature 0.

plaintext

agreement across 5 runs of one version (example counts):
identical replies: 4 -> 4 / 5 = 80%
set score after the edit: 3 of 6 hold, up from 2 of 6
temperature 0 is approximately greedy, not identical: rerun before judging an edit

Common mistakes

Check yourself Version 5 adds two facts and a format example at once, and the set improves. What can be concluded?

Little. Two edits confound the cause, so each edit needs its own rerun.

Check yourself Five runs of one version at temperature 0 give 4 identical replies. What does that show?

Agreement of 4 of 5, or 80 percent. Temperature 0 is approximately greedy rather than identical, so rerun before judging an edit.