Large language models are billed by the token, and that bill has a way of growing quietly until it surprises someone in finance. The good news is that most LLM cost is controllable without sacrificing quality — often the cheaper design is also the faster and more reliable one. Here are the levers that matter.
Understand where the cost actually goes
Before optimising, measure. LLM cost is driven by the number of tokens processed — both the input you send and the output the model generates — multiplied by the price of the model you use. A surprising amount of spend often comes from sending far more context than necessary, calling a larger model than the task requires, or repeating work that could have been cached. You cannot cut what you have not measured, so start by understanding which calls dominate your bill.
Why costs surprise people
The reason LLM bills catch teams off guard is that the per-call cost feels trivial. A single request costs a fraction of a cent, so during development nobody thinks about it. But production runs the same call thousands or millions of times, and small inefficiencies multiply accordingly. A pipeline that sends twice as much context as it needs is not wasteful at demo scale; at production volume it doubles a significant bill. The fix is to think about cost per call multiplied by realistic volume from the beginning, rather than treating it as a problem to address later.
Right-size the model
The most common source of waste is using an expensive, powerful model for tasks a smaller one would handle perfectly. Classification, extraction, simple formatting and routing rarely need the largest model available. Matching model size to task difficulty — a small model for simple steps, a large one only where genuine reasoning is required — can cut cost dramatically. Many pipelines can route easy cases to a cheap model and reserve the expensive one for the hard minority.
Trim the context you send
Every token of input is billed, and teams routinely send more context than the task needs — entire documents where a relevant section would do, long histories where a summary would suffice. Retrieving and sending only the passages that matter, rather than everything that might, cuts input cost and often improves quality by reducing noise. Tighter context is usually better context, so this is a rare optimisation with no downside.
Cache aggressively
Many applications ask the same or very similar questions repeatedly. Caching responses to common queries avoids paying for the same generation again and again, and it makes the system faster for users at the same time. Even partial caching — of retrieved context, of intermediate results — reduces the work each request requires. For any application with repeated patterns, caching is one of the highest-return changes available.
Control output length
Output tokens are billed too, and models left unconstrained often produce more than needed. Being explicit about the length and format you want — a short answer, a specific structure — reduces output cost and usually makes the result more useful. A model asked for a concise, structured answer is cheaper and easier to consume than one allowed to ramble.
Batch where you can
For work that does not need an immediate response — overnight processing, bulk classification, large-scale enrichment — batching requests is often substantially cheaper than handling them one at a time in real time. Separating the genuinely interactive work from the work that can wait lets you use cheaper processing modes for the latter, which is frequently the bulk of the volume.
Monitor cost like you monitor errors
The final lever is visibility. LLM cost tends to creep because nobody is watching it until the invoice arrives. Treating cost as an operational metric — tracking token spend per feature, alerting when it jumps, attributing it to the calls responsible — turns a quarterly surprise into a managed number. The same instinct that makes teams monitor error rates and latency should extend to spend, because with usage-based billing, a bug that doubles token consumption is as real an incident as one that doubles error rates. Cost observability is cheap to add and repeatedly pays for itself.
The quality trade is usually a myth
The reassuring truth is that cost control and quality usually point the same way. A right-sized model, tight context, controlled output and caching produce a system that is not only cheaper but faster and often more reliable, because it does less unnecessary work. The teams that treat cost as a design constraint from the start tend to build better systems, not worse ones — the discipline of not wasting tokens is the same discipline that keeps a system focused and predictable. None of these levers requires sacrificing what makes the system useful; they simply remove the waste that accumulates when nobody is paying attention. Applied together — right-sizing, trimming context, caching, controlling output, batching and monitoring — they routinely cut a bill by a large margin while leaving the user experience untouched or improved. That is the rare kind of optimisation worth doing early and revisiting often.