LLM API costs higher than your AI feature earns: cut, reprice or cap?

When LLM API costs run above what the AI feature earns, the fix is usually in the request, not the price list. Compute cost per active user, then pull levers in order: cache and shorten the request, route routine calls to a cheaper model, batch work nobody waits on, cap the heaviest users, and reprice last. Give one engineer the line and a 30-day review.

(This is about your product's costs, not the team's own AI coding bill.)

Is the feature actually losing money, and on whom?

Cost per active AI user is requests per month times the cost of one request. A request costs its input tokens times the input price, plus its output tokens times the output price. Pull that from your vendor's usage export, grouped by user, and look at the shape.

If most users sit above what they pay, the loss is the whole base, and engineering or price has to close it. If a few sit far above and the rest are fine, you have a heavy tail, and a cap does most of the work. ICONIQ's 2026 State of AI survey, as The SaaS CFO reports it, has AI product builders expecting about 52% gross margin, so a thin margin is common. A negative one isn't something to wait out.

Here's a workload to price. An AI research assistant add-on in a B2B SaaS sells at $15 per user per month. The usage figures are illustrative assumptions, so swap in your own. Each active user sends 150 requests a month. Each request has 40,000 input tokens and 3,000 output tokens. Of that input, 30,000 tokens are a fixed prefix of instructions and tool definitions, and 10,000 are new content.

Prices checked 6 November 2026: Anthropic's pricing page lists Sonnet 5.5 at $2 per million input tokens and $10 per million output tokens. The input costs $0.08 per request. The output costs $0.03. Each request costs $0.11. Over 150 requests that's $16.50 per user, which is $1.50 more than the add-on earns, a margin of -10%.

What does the same feature cost after each fix?

The 80% hit rate and the 70% routable share below are assumptions; your usage fields and your eval supply the real ones.

Scenario	Cost per user	Margin on $15
Cache the prefix, 80% hit rate	$10.11	$4.89 (33%)
Cache the prefix, 50% hit rate	$13.35	$1.65 (11%)
Route 70% of requests to Haiku 5.5	$5.53	$9.47 (63%)
Batch the 20% nobody waits on	$14.85	$0.15 (1%)
Route 70%, cache the rest at 80%	$3.61	$11.39 (76%)

Batching does little here and routing does a lot. Where the saving comes from depends on your request mix, not on which lever sounds clever.

Cut the cost in engineering: cache, route, batch

This is the first lever because it changes what each request costs without touching a customer. It has three parts.

Caching suits a request where most of the prompt repeats, as the 30,000-token prefix does here. Without caching, that prefix costs $0.06 per request. A cache read costs $0.10 per million tokens, so reading the prefix costs $0.003. A cache write costs 1.25 times the input price, so writing it costs $0.075.

At an 80% hit rate, four requests in five pay the read and one pays the write. That averages $0.0174 for the prefix, instead of $0.06.

The catch is that the prefix must match exactly, the cache lasts five minutes unless use refreshes it, and changing a tool definition invalidates everything after it. Watch cache_read_input_tokens in the response's usage block to see your real hit rate. Separately, output tokens cost five times what input tokens do, so trimming a 3,000-token answer pays too.

Routing suits a workload with many simple requests, such as classification or extraction. A Haiku 5.5 request of the same size costs $0.0055, against $0.11 on Sonnet. Hacker News commenters call the top model wasteful for mechanical work, and one company's post there describes a cheap model that triages and escalates only the hard cases. The catch is quality, which the eval below measures. Haiku 5.5 also charges five times more for input once a prompt passes 100,000 tokens, which doesn't apply at 40,000 (see Haiku 5.5's per-prompt price tiers), but does if your prompts are longer.

Batching suits work nobody is waiting on, like nightly digests. Anthropic's batch API takes 50% off input and output, and most batches finish within an hour. A batch not finished in 24 hours expires, so it can't carry anything with a deadline shorter than a day.

Reprice the plan

Repricing suits the case where engineering has done what it can and a gap remains, or where the whole base is underwater and price is the faster move. Work backwards from cost per user. A 70% margin at the baseline cost needs $16.50 divided by 0.30, which is $55 per user, 3.7 times today's price. That's why engineering comes first. After the routing and caching row above, the same margin needs about $12, below what you already charge.

The catch is that it's a sales and contract decision, not an engineering one. Customers on annual terms may keep today's price until renewal, so read your contracts before promising finance a date.

Cap or meter the heaviest users

A cap suits a heavy tail. In the example, a user at 600 requests a month costs you $66 at baseline against $15 of revenue. Include 150 requests in the plan and meter the rest at a price above your cost per request, and the tail pays for itself. Agent loops make this worse because their request counts are hard to predict, so cap loop iterations per task as well as requests per month.

After the engineering row above, that same 600-request user costs about $14, roughly break-even. Caps matter most while the cost per request is still high, and they matter less once engineering has lowered it.

The catch is that a cap is customer-facing. It needs a metered allowance in the plan and overage wording in your terms. Put it in front of the heaviest tenth of users first and watch support tickets before rolling it out. Your own backstop is different: set a spend limit in the Console, so a bug or a runaway loop can't run up the bill.

What to measure before you commit

Three things only your own traffic can tell you.

Does the cheaper model hurt quality on your task? Replay a sample of real requests through Haiku 5.5 and score the output against what you ship today. Add a per-request-type switch so you can roll back one workload without the rest.
How many engineering days does each lever take? Time-box a one-week spike on caching, the lever with the least to change, and use what it took to estimate the rest.
What share of users forms the tail? Group the usage export by user and count how many sit above what they pay.

What this means for your AI budget

Make the AI line a variable cost with a ceiling per plan, one named owner and a review at the next invoice. The same feature costs $16.50 or $3.61 per user depending on request design, so a budget without an owner will drift between the two.

Name one engineer who owns cost per user and one person in finance who owns the price. Give the engineer the usage export, the table above rebuilt with your numbers, and a 30-day review that asks one question: did the cost per active user fall, and by how much? If you decide to run the feature at a loss as acquisition spend, give it a ceiling and an end date, and revisit it at the same review. Our software development cost guides cover the other lines in the same budget.

Getting a senior engineer to own the cost work through HighCircl

If the cost line has no owner with time to take it on, the review date will pass without a change. HighCircl's hiring service sends a shortlist within 72 hours. There's no minimum hour commitment, which suits this work: the spike is a week and the review is 30 days, so you can scope the engagement to that stretch instead of committing to a minimum number of hours.

FAQ

Why are my LLM API costs so high?

Because per-user usage times the token price is above per-user revenue. In practice the cause is usually a long prefix sent uncached on every request, or the top model handling routine calls. Group your usage export by user to see whether it's the whole base or a heavy tail.

How do I calculate LLM cost per user?

Multiply input tokens by the input price and output tokens by the output price to get the cost of one request. Multiply that by the user's requests per month. In the example above, that's $0.11 per request and $16.50 per user at baseline.

Does prompt caching reduce LLM costs?

Yes, when the prompt starts with a long, identical prefix. A cache read on Sonnet 5.5 costs $0.10 per million tokens against $2 for fresh input, but the first write costs 1.25 times the input price and the cache lasts five minutes. The saving depends on your hit rate, which the usage fields report.

Is the batch API cheaper?

It takes 50% off input and output, for requests that can wait. Most batches finish within an hour, and a batch that isn't done in 24 hours expires. It only helps the share of your traffic nobody is waiting on.

Should I raise prices or cut costs first?

Cut costs first when the cost per request is high, because engineering can change it without a customer noticing. Reprice when a gap remains after that, or when every user is underwater. Cap the heaviest users if the loss comes from a tail.