Most bad replies trace to the prompt rather than the model. Debugging a prompt means classifying the failure, changing one thing, and rechecking a small fixed set of test prompts.
Symptom | Likely class | First test |
|---|---|---|
Reply invents facts the prompt never gave | Missing context | Add the missing facts and rerun. |
Reply follows a different reading than intended | Ambiguity | Define each key term once and rerun. |
Right content in an unusable shape | Wrong format | Pin the shape with one example. |
Failure survives every phrasing | Capability limit | Split the task or check the step another way. |
Lesson | What it covers |
|---|---|
Four failure classes and the test that tells them apart. | |
Single-variable edits with a before-and-after log. | |
Test prompts with checkable expected properties. | |
A runnable Python harness over a scripted fake model. |
Wrong format. Pin the shape in the prompt, for example by naming the columns once.