How to estimate AI API costs before shipping

Start with a formula

Estimate usage for each AI feature before launch. When rates are quoted per million tokens, multiply the total token count by the rate and divide by 1,000,000. Calculate input and output separately, then add tool, image, audio, storage and other applicable charges. Check the provider's current units; the OpenAI API pricing page lists model and feature rates. Keep the date checked beside each rate in your worksheet.

Measure tokens from real examples

Do not estimate token counts from vibes. Collect twenty realistic requests: short, average, long and abusive. Include system instructions, developer instructions, user input, retrieved context, tool outputs and expected response length. If your app uses chat history, include a realistic history window. If it uses RAG, include retrieved chunks. If it returns JSON, include the whole output. The OpenAI models documentation is where you identify candidate models, but your logs or test harness should supply the token sizes.

Example spreadsheet

Account for retries and validation

A structured extraction feature may call the model again when JSON is invalid. A search assistant may rerun retrieval with a better query. A voice agent may create many short turns. Add a retry multiplier. A conservative first estimate might compare three cases: expected, heavy and worst ordinary day. Label the assumptions clearly. If a feature is user-facing, also estimate abuse: very long pasted text, repeated refreshes and batch usage through automation.

Prompt caching changes the shape

Prompt caching can reduce cost or latency when a supported model sees the same long prefix across requests. OpenAI's prompt caching guide explains that caching works on matching prompt prefixes for supported models and exposes cache-related usage fields. The planning lesson is to separate stable context from per-request content. System instructions, long policy text and tool schemas may be stable. The user's question and retrieved evidence usually are not. Caching does not make unnecessary context free, and cached tokens may still count against throughput limits, so design for smaller prompts first.

Problems to check

• Ignoring output: teams price input context and forget long generated reports.
• Forgetting tools: web search, image generation, audio or code tools have separate economics.
• Average-only planning: a few long requests dominate monthly cost.
• No budget guardrail: users can submit unlimited long tasks.
• Stale rates: a spreadsheet survives after pricing and models change.

Next steps

Create a cost worksheet before implementation is finished. Add a line for each user-visible AI action. Use measured token counts from prototypes, current official rates, retry assumptions and growth scenarios. Then add product limits: maximum input length, maximum output length, monthly budget alert, per-user throttles and a fallback path when a cheaper model is good enough. Compare the worksheet with observed usage after launch and investigate features that exceed their assumptions.

Budget guardrails

Turn the worksheet into product controls. Add input length limits, output length limits, per-user monthly caps, admin alerts and graceful failure messages. For internal tools, show estimated cost before a large batch starts. For customer features, protect the system from repeated refreshes and accidental loops. Cost control works best when users understand the boundary before they hit it.

After launch

Compare estimates with real usage during the first week. Look for long-tail requests, retry loops, unusually large outputs and users who batch more than expected. Update the worksheet with actual percentiles, not only averages. The second estimate is usually the one that becomes reliable.

Procurement note

If finance needs approval, present ranges instead of a single number: expected, heavy and capped maximum. A range makes assumptions visible.

A worked example with hypothetical rates

Suppose a feature handles 10,000 requests per month, averaging 3,000 input tokens and 500 output tokens. For illustration only, assume input costs $1 per million tokens and output costs $4 per million tokens. These are invented planning rates, not a provider quote.

If retries add 10% with the same average token sizes, the token estimate becomes $55. Add separate tool and storage charges afterward. For caching, split input into the billing categories your chosen model uses, such as uncached input, cache reads and cache writes. Current OpenAI documentation distinguishes these categories; a cache-write rate applies to those tokens rather than being an extra fee on top of their input rate. Replace every illustrative rate with a checked quote before budgeting.

Sources and further reading

Documentation checked October 4, 2026. Product features and availability can change.

• OpenAI API pricing
• OpenAI models documentation
• OpenAI prompt caching guide