Reducing Tokens · ai-skills

Reference: Tokens, Cost and Latency · Beginner to Intermediate

4topics
4lessons

Most of a long prompt is repeatable background, and only a slice of it decides the next answer. Four techniques shrink the rest: reuse identical prefixes, summarise history, retrieve a few chunks, and trim logs and RTL to the failing slice.

Technique	What it saves	Main risk
Reuse identical prefixes	Repeat tokens across calls with the same opening	Any edit to the opening can end the reuse.
Summarise history	Long background and old turns	Details that later questions need are dropped.
Retrieve top chunks	Whole manuals pasted for one fact	The best chunk may rank too low to be included.
Trim to the failing slice	Passing bulk of logs and unrelated files	The error itself gets cut when the window is too tight.

Flow from full context through reduction to a short prompt and its answer.

Each technique removes a different kind of surplus before the short prompt is sent.

Lesson	What it covers
Prefix caching	What stays identical can be reused, with a worked saving.
Summarisation and information loss	Compression ratios and the detail a summary drops.
Retrieval instead of pasting	Chunking and keyword ranking with a runnable TF-IDF toy.
Trimming logs and RTL	Failing slices of logs and RTL, plus cost against quality.

Check yourself A second call repeats the first call's long specification word for word, then asks a new question. Which technique applies, and what must stay unchanged?

Prefix reuse applies. The specification opening must stay byte-identical for the repeat to count as reuse.

Up nextPrefix Caching

Practice this lesson Report an error

Technique

What it saves

Main risk

Reuse identical prefixes

Repeat tokens across calls with the same opening

Any edit to the opening can end the reuse.

Summarise history

Long background and old turns

Details that later questions need are dropped.

Retrieve top chunks

Whole manuals pasted for one fact

The best chunk may rank too low to be included.

Trim to the failing slice

Passing bulk of logs and unrelated files

The error itself gets cut when the window is too tight.

Lesson

What it covers

Prefix caching

What stays identical can be reused, with a worked saving.

Summarisation and information loss

Compression ratios and the detail a summary drops.

Retrieval instead of pasting

Chunking and keyword ranking with a runnable TF-IDF toy.

Trimming logs and RTL

Failing slices of logs and RTL, plus cost against quality.