Stop chatting, write one prompt — Using LLMs Well (5)

At some point you start treating the agent like a messaging app. "Fix this bug." Enter. "Oh, and check the logs too." Enter. "No, not the UI — the session part." Enter. It's the same habit you use texting on WhatsApp: fire whatever comes to mind, one line at a time. It's convenient. And that convenience isn't free.

This isn't about manners. A companion piece covered the cost of making a single request vague. This is a different axis entirely — the cost of splitting one task across many messages. Even when every line is perfectly clear, breaking them apart leaks tokens. The reason lives in how an LLM remembers a conversation.

What actually happens

An LLM has no memory. The model is stateless. Every time you send a message, it re-receives the entire conversation so far, from scratch — the system prompt, your earlier messages, its own earlier replies, and every tool output in between. On screen, only the one line you just typed is added. Inside, everything in front of that line is re-processed as input tokens.

So the real cost of one more message isn't its length. It's "that message plus everything before it." The longer the conversation, the heavier that tail becomes.

Why — every turn is recomputed in full

First, the fixed overhead is re-billed every turn. A coding agent always carries a load: a long system prompt, tool definitions, standing project instructions like CLAUDE.md. That alone is thousands of tokens, and it goes in on every turn regardless of what you said. Split one task into eight messages and you pay that fixed cost eight times.

Concretely, say the fixed overhead is 5,000 tokens (often it's more).

• In one message: one turn. Overhead 5,000 + your complete request 400 ≈ 5,400 tokens, computed once.
• Fragmented: the same request as eight 50-token lines. The overhead reloads each turn — 5,000 × 8 = 40,000 tokens for the overhead alone, plus the prior messages and replies piling onto every turn after.

The numbers are simplified, but the direction is unambiguous: the cost isn't driven by the extra 50 tokens you typed. It's driven by the thousands of tokens of context dragged along each time.

Second, each fragment gets a reply. Eight messages means eight model responses — many of them "Got it," "What next?" — replies you never needed. Output tokens usually cost more than input. Eight throwaway responses add up.

Third, the agent re-orients on each fragment. Responding to a piece, the agent re-opens files, re-plans, and re-establishes context it would have held in one pass had you given it the whole picture. That rework becomes tokens too — which then join the tail of the next turn.

How far prompt caching helps (and where it breaks)

Here's where honesty matters. Modern APIs have prompt caching: the unchanged prefix (system prompt, tool definitions) is reused cheaply — around 10% of the cost — on a cache hit. So if you keep turns fast and continuous, that "overhead × 8" is largely clawed back.

The catch is the cache's lifetime (typically ~5 minutes). And the real pattern when we use an agent like a messenger — type a line, get distracted, come back a few minutes later, change your mind mid-thread — is exactly what breaks that cache most often. Cross the five-minute gap or edit earlier content, and the next fragment recomputes the whole context at full price. Caching also only applies to reused input; the second and third costs above — the per-fragment output and the agent's rework — don't shrink with it.

In short: caching rescues a fast, unbroken conversation. It does not rescue drip-fed commands sent whenever something pops into your head.

The bigger cost is actually quality

It isn't only tokens. Fragmenting makes the model act on an incomplete intent. It commits to a direction on half the information, then the next fragment changes a premise, and it backtracks. The moment "no, not that" arrives, part of the work is thrown away and you've added another round-trip — each one stacking more of the recomputation above. A single complete instruction gives the model the whole picture from the start, and most of those round-trips never happen.

So how should you write it — one complete prompt

Before you hit send, gather the scattered request into one block. Five things usually cover it:

• Goal — what you're trying to achieve.
• Context — the relevant files, symptoms, constraints.
• Expected result — what "finished" looks like.
• Out of bounds — what must not be touched.
• Done criteria — how you'll judge that it's done.

Put a bad and a good version side by side and the difference is obvious.

❌ Fragmented:

"fix the login bug" "oh when the session expires" "the 401 that keeps repeating" "no not the UI, the middleware" "and run the tests too"

✅ All at once:

"When a session expires, the app loops on 401. In auth.middleware.ts, detect the expired cookie and route to the re-login flow. Don't touch the login UI, and after the fix, run the related tests and confirm they pass."

The same information, sent in one message, computes its context once. Scattered across five, that context is recomputed five times, five replies are generated, and the agent re-orients five times. The total input looks similar — the tokens the model actually processes are in a different league.

RoutineCode is built to keep that tail light

Give RoutineCode a complete instruction and it classifies the intent once and routes the whole task — rather than taking fragments and re-orienting on each. The more complete your instruction, the more that design pays off.

The difference also shows in the weight of the recomputed tail. Routine lookups — "does this build," "where is this symbol used," "how do the dependencies connect" — a stock agent answers by opening files into context and reasoning over them, and those file bodies then ride along in the tail of every subsequent turn. RoutineCode handles those lookups with deterministic tools and returns a structured answer instead of the file body. With a lighter tail, even a multi-turn session recomputes far less each turn. When broad exploration is genuinely needed, it pushes that cost into an isolated sub-agent window and protects the main conversation's budget.

You consolidating your request and RoutineCode keeping its tail light are two directions toward the same goal: keep the context that gets recomputed every turn as small as possible.

What you get

• Fewer tokens for the same work. Heavy context isn't recomputed over and over.
• The right result in one shot. The model holds your full intent from the start, so the "no, not that" round-trips drop.
• Budget that lasts in long sessions. A light tail fills the context window more slowly.

Using an agent well comes down to giving it good input. And good input has two rules — aim the request narrowly (part one) and say it all in one complete pass (this one). There's no need to carry your messaging habits into how you give instructions.

---

This is part of the "Using LLMs Well" series. Read it alongside part one, Precise requests aren't politeness — they're token economics, and you get the pair: how what you ask and how you split it up each turn into tokens.

References

• Prompt caching — Anthropic
• Explore the context window — Anthropic
• Best practices for Claude Code — Anthropic