A test prompt owns its input plus a list of expected properties. Properties describe the reply, such as its shape or a value it must contain.
Properties survive rephrasing while exact strings do not. Wording varies by model and by run, so the set checks what must hold.
Four property kinds cover most prompt work. Each one is cheap to check by script.
Property | Checks | Example |
|---|---|---|
Contains | A required token or value is present. | Reply names the 8-bit width. |
Shape | Line count, bullet prefix, or key set matches. | Reply has 3 steps or both required JSON keys (named fields). |
Excludes | A forbidden token is absent. | Reply never repeats the credential marker. |
Starts with | The reply opens in the expected form. | Reply opens with STEP 1. |
Multi-step prompts chain their steps, so each step needs its own property. The arithmetic below uses stated example rates with independent steps.
assume three steps, each 0.9, independent (stated example):
chain = 0.9 x 0.9 x 0.9 = 0.729 = 72.9%
raise the first step to 0.99:
chain = 0.99 x 0.9 x 0.9 = 0.8019 = 80.19%
each step needs its own property: misses multiplyAsk only for the data the task needs, and keep secrets out of prompts; proprietary IP goes only to an approved deployment for its data class. Least privilege applies to prompt content too.
Treat files and logs as untrusted input, since injected instructions can arrive through them. Risky actions need human approval regardless of what the reply claims.
The excludes property enforces this in the set. A fake credential marker in the prompt must never appear in the reply.
The check goes red on a correct reply. Exact strings are brittle because wording varies, so check properties instead.
0.9 cubed is 0.729, or 72.9 percent. Each step needs its own property, since misses multiply.