Home › Guides › Cut your bill without switching models

How to cut your LLM API bill without switching models

Switching models means re-testing quality, re-tuning prompts and re-writing evals — so most teams don't, and overpay instead. The good news: the model is rarely the problem. The same model, called the same way, usually gets 30–60% cheaper with the seven levers below, ordered from least to most effort. None of them change which model answers.

1. Turn on prompt caching (biggest lever, near-zero effort)

If your requests share a stable prefix — a system prompt, tool definitions, a document, few-shot examples — you are probably paying full input price for the same tokens thousands of times a day. Every major provider offers a cached-input rate typically 5–10× cheaper than fresh input; the model pages here list it as the "cached" column. Two rules make it work: put the stable content first and the variable content last, and don't let timestamps or request IDs leak into the prefix, because one changed byte can invalidate the whole cached block. This is the same mechanism explained in how LLM API pricing works — it is first on this list because for chat and agent workloads it routinely halves the input bill on its own.

2. Move everything non-urgent to the batch API

Major providers run batch endpoints — requests processed within hours instead of seconds — at roughly 50% off list price. Evals, data labeling, document classification, nightly summarization, embedding backfills: none of these need a synchronous answer. A useful audit: tag every workload in your system "user is waiting" or "user is not waiting". In our experience the second bucket is larger than teams expect, and it is the easiest discount you will ever collect.

3. Stop paying for output you throw away

Output tokens are the expensive side (commonly 3–4× input, sometimes 10×). Three habits burn them for nothing:

No max_tokens cap. Set one per endpoint. A runaway generation should hit your cap, not your credit card.
Prose you parse and discard. If code extracts a JSON field from a paragraph, ask for structured output directly — shorter, cheaper, and more reliable to parse.
Unbounded politeness. "Answer in at most three sentences" or an explicit format template measurably shortens replies. Verbose is a setting, not a personality trait.

4. Budget reasoning like it costs money — because it does

Reasoning models bill their internal thinking tokens at the output rate. Most providers expose an effort or thinking-budget setting, and the default is rarely the right one for every request. Route by difficulty: classification and extraction usually need the lowest setting; hard debugging earns the highest. Prove it with a small eval per tier instead of guessing — teams routinely find their mid setting matches their max setting on 80% of traffic.

5. Shrink the context you resend

Every token in the prompt is billed on every request. Long chat histories resent verbatim, giant system prompts, and whole documents stuffed in "just in case" are a per-request tax. Summarize history past a threshold, retrieve the relevant chunks instead of attaching the whole file, and dedupe few-shot examples that stopped teaching anything months ago. A 40% shorter prompt is a 40% cheaper input line — forever, on every call.

6. Cache responses at your layer

The cheapest API call is the one you never make. Exact-match caching (same prompt, same parameters → same stored answer) fits FAQ-style and deterministic workloads; even a modest hit rate compounds. Log your duplicate rate before building anything fancy — many teams find 10–20% of requests are literal repeats.

7. Measure, then hunt the whales

None of the above sticks without numbers. Log input/output/cached tokens per endpoint and per feature for two weeks, then sort by cost. The distribution is never flat: a small share of request types — one chatty agent loop, one endpoint with a bloated prompt — usually accounts for most of the bill. Fix the top three and re-measure. Use the cost calculator to sanity-check what a workload should cost at list prices, and watch the daily price changes: when your model's rate drops, your bill should drop with it — if it doesn't, something on your side is off.

The checklist

Prompt caching on, stable prefix first — target the 5–10× cached rate.
Batch API for every "user is not waiting" workload — ~50% off.
max_tokens everywhere; structured output instead of parse-and-discard prose.
Reasoning effort tiered by task difficulty, verified by evals.
History summarized, retrieval instead of stuffing, prompts deduped.
Response cache for repeat requests.
Per-feature token logging; fix the top three cost whales quarterly.

Do these and the model debate gets quieter: at a 40% lower bill, last quarter's "expensive" model is often this quarter's sensible default.

Next: how we verify the prices on this site — and where price data goes wrong →

By the LLMPrices editorial team · October 2026. All pricing guides →