The Unit Economics of AI Features: Token Budgets, Latency, and Margins

Why the unit economics of AI features need their own model

A conventional SaaS feature costs roughly nothing to serve once it exists. An AI feature costs money every time a customer uses it, and the amount varies with the input, the model, and how the feature was built. That single difference is why the unit economics of AI features deserve their own model, and why teams that skip it discover margin problems only after the feature becomes popular. This post covers the levers we use when building AI features for clients and for our own products, with a worked example at the end.

Cost per task, not cost per request

The first mistake is to price by API call. What a customer buys is a completed task: a screened resume, a summarised call, a drafted reply. A task may take one request or twelve, with retries, tool calls, and a verification pass, and the cheaper-looking model may need more of them.

So the unit we track is cost per completed task, measured from production usage logs: total tokens in and out (split into cached and uncached input), across every request that contributed to the task, divided by the tasks that actually finished. A model with a higher list price that completes the task in fewer, shorter turns is often the cheaper option. Anthropic's guidance on effort levels makes the same point: judge cost per completed task, because request-level comparisons mislead.

Two derived numbers matter for the business side:

• Cost per task at p50 and p95. The long tail — huge documents, pathological inputs — is where budgets break.
• Cost per active customer per month, which is what you compare with the price.

Three levers that change the input bill

Prompt caching

Most AI features have a large stable prefix (system prompt, tool definitions, reference material, a long document) and a small varying suffix (the user's question). Prompt caching lets the stable prefix be reused across requests; cached reads are billed at a small fraction of the normal input price, with a modest premium on the initial write. For a feature that sends the same 20,000-token context on every turn, this changes the input side of the bill dramatically.

The engineering discipline is prefix stability: nothing that varies per request, including timestamps and request IDs, may appear before the cache breakpoint. We watch cache_read_input_tokens in the usage data in production; if it is zero on repeated requests, something is silently invalidating the cache.

Batch processing

Anything that does not need an answer within seconds — nightly reports, bulk classification, re-scoring a backlog — should go through batch processing, which is priced at half the standard rate in exchange for asynchronous processing. The design work is to identify which parts of a feature are genuinely latency-sensitive and which merely feel that way because they were built synchronously.

Model and effort selection

Model choice is the most visible lever and often the least well-measured. The pattern we use is routing by task difficulty: a small, fast model handles classification and extraction; a strong model handles synthesis and anything customer-facing that needs judgement. Within a model, the effort setting trades reasoning depth for tokens and time; for many routes, a capable model at low or medium effort holds quality and cuts both cost and latency. Measure on a sample of real tasks before changing defaults, and treat the current pricing page as the source of truth rather than a number remembered from last quarter.

Latency budgets and what the user feels

Cost and latency are the same problem seen from two sides: more tokens means more money and more waiting. We set a latency budget per feature the same way we set a cost budget, and the budget decides the architecture.

• Under about a second: the feature needs to feel instant. Only cached, short prompts on fast models qualify; often the right answer is to precompute.
• A few seconds: acceptable with streaming, so the user sees progress. Most interactive features live here.
• Tens of seconds or more: run in the background and notify when done. Do not make the user watch a spinner.

The Nielsen Norman Group has published response-time guidance for decades, and the thresholds at which people stop feeling in direct control, lose their train of thought, and give up are still the ones to design around. Streaming buys a lot: a visible first token in under a second keeps an interaction feeling responsive even when the full answer takes ten.

Pricing AI features for SaaS

Three models cover most of what we see:

1. Bundled into the plan. Simple for the customer; you carry the variance. Works when cost per active customer is small relative to the plan price and usage is not too skewed. Set a fair-use cap you actually enforce.
2. Metered usage or credits. Cost tracks revenue; customers have to think about spend. Works for high-variance features where a few heavy users would otherwise dominate cost.
3. Premium tier. AI features gate a higher plan. Works when the AI capability is the reason to upgrade and heavy users are the ones who will pay.

Whichever you choose, the internal rule is a target gross margin per feature, and a cap on cost per task at p95 that keeps you inside it. In the Indian SaaS market, where plan prices are often lower than in the US but usage is comparable, the margin question is sharper and usually pushes toward metering or a premium tier rather than bundling.

Monitoring spend

What gets measured gets controlled. The minimum instrumentation:

• Log every request with model, cached and uncached input tokens, output tokens, latency, and the feature and customer it belongs to.
• Roll up to cost per task and cost per customer daily; alert on a customer or feature whose cost deviates from its baseline.
• Track cache hit rate per feature. A sudden drop is usually a deploy that broke prefix stability.
• Add hard budgets: a per-customer daily cap and a per-task ceiling on tokens and turns, so a runaway loop cannot become a surprise invoice.

The Google SRE book offers a useful frame for the alerting side: alert on symptoms customers would feel (task failures, latency) and on cost that threatens margin, not on every fluctuation.

A worked example (illustrative numbers)

The numbers below are illustrative — chosen to show the arithmetic, not taken from any real invoice or price list. Check the current pricing page for actual rates before modelling your own feature.

Suppose a feature summarises recorded interviews. Each task sends a 3,000-token system prompt and rubric (stable), a 15,000-token transcript (varies per task), and produces a 1,500-token summary. Assume illustrative rates of $3 per million input tokens and $15 per million output tokens for the strong model, $1 and $5 for a small model, cached reads at a tenth of the input rate, and batch at half of everything.

Caching alone is a modest gain here because the transcript, not the prompt, dominates the input. The two-model pipeline helps because the strong model no longer reads the whole transcript. Batch halves whatever is left, but only if recruiters do not need the summary live.

At 20,000 interviews a month, the naive path costs about $1,530 and the batched pipeline about $460, again on illustrative rates. Whether that matters depends on price. If each interview is billed separately, it is noise. If the feature is bundled into a plan where each recruiter runs hundreds of interviews, it is the difference between a healthy margin and a loss-making feature. The point of the table is the method, not the figures: model each configuration, then measure the real one.

The habit that matters

Every AI feature we ship now has a one-page economics sheet before it reaches production: the task definition, the latency budget, the cost per task at p50 and p95 on real samples, the caching and routing decisions, and the pricing model it sits under. We revisit it when model prices change, which they do. The unit economics of AI features are not hard to model, but they are easy to ignore until the feature is popular, and by then the pricing page is already public.

Further reading

• Prompt caching in the Claude documentation
• Batch processing
• Effort levels
• Current model pricing
• a16z: Emerging architectures for LLM applications