Most of what you pay for when an AI chatbot answers a customer is text the customer never sees. Each new message sends the instructions and the whole conversation so far back to the model, and you pay to have all of it read again. In the example below, that re-reading is more than 80% of the bill.
Per-token prices are easy to find. Turning them into a monthly number is where estimates go wrong, so this piece works through one realistic support chat at three vendors’ published prices, then shows the levers that actually move the total.
A note on method: this is arithmetic on published price lists, not a measured bill. We did not run this chatbot. The token counts are assumptions sized for a short support conversation, and the calculation is kept so you can rerun it with your own numbers.
What you are actually paying for
A token is a chunk of text, often part of a word. Anthropic’s rule of thumb is that one token is roughly four characters or 0.75 English words, though it varies by language and content.
API bills have two main meters:
- Input tokens: everything you send in a request. That means your instructions, any reference material, the conversation history and the new message.
- Output tokens: everything the model generates, including reasoning you may never see. Output is priced several times higher than input at all three vendors below.
The model keeps no memory between requests. If you want it to know what was said three messages ago, your software sends those three messages again, and they are billed again.
The prices, checked on 13 September 2026
Standard rates per million tokens for one mid-priced model from each vendor, taken from Anthropic’s pricing page, OpenAI’s pricing page (short-context rates) and Google’s Gemini pricing page (updated 11 September).
| Model | Input | Cached input (read) | Writing to cache | Output |
|---|---|---|---|---|
| Claude Sonnet 5 | $2.00 | $0.20 | $2.50 (5-minute cache) | $10.00 |
| GPT-5.6 Terra | $2.00 | $0.20 | $2.50 | $12.00 |
| Gemini 3.8 Flash | $0.75 | $0.075, plus hourly storage | not listed | $3.75 |
Google lists the Gemini 3.8 Flash rates as running through 31 December 2026, rising to $1.50 input and $7.50 output from 1 January 2027. Prices in this market change often, so check the pages above before you budget.
Worked example: one support chat, priced three ways
The setup. A small business puts a chatbot on its website. Every request carries 3,000 tokens of instructions and FAQ content. A typical conversation runs 10 exchanges, with each customer message about 60 tokens and each reply about 150.
The estimate most people make. Count the visible conversation: 10 messages in, 10 replies out, 2,100 tokens. At the prices above, 1,000 conversations would cost $16.20 on Claude Sonnet 5, $19.20 on GPT-5.6 Terra and $6.07 on Gemini 3.8 Flash.
What actually gets sent. The first request is 3,060 input tokens: the instructions plus one message. The second carries the instructions, the first exchange and the new message, 3,270 tokens. By the tenth request it is 4,950. Across the conversation that adds up to 40,050 input tokens to produce 1,500 tokens of replies.
| Scenario, per 1,000 conversations | Claude Sonnet 5 | GPT-5.6 Terra | Gemini 3.8 Flash |
|---|---|---|---|
| Visible text only (the naive estimate) | $16.20 | $19.20 | $6.07 |
| What is actually billed | $95.10 | $98.10 | $35.66 |
| Plus 500 hidden reasoning tokens per reply | $145.10 | $158.10 | $54.41 |
| Billed, with prompt caching on every turn | $34.40 | $37.39 | not modelled |
The real bill is five to six times the visible-text estimate in every column, and on Claude, $80.10 of the $95.10 is input. The 500 reasoning tokens are an illustrative figure, not a measured one; the real amount varies widely by model, setting and question. We left Gemini out of the caching row because its cached tokens also carry an hourly storage charge, which depends on how you set the cache up.
Hidden reasoning is billed as output
Many current models work through a problem before writing the reply. OpenAI’s documentation says those reasoning tokens are not visible, still take up context window space, and are billed as output tokens. Anthropic bills Claude’s thinking as output tokens, and thinking from earlier turns that stays in the conversation is billed again as input. Google’s price list says its output price includes thinking tokens.
Because output is the expensive meter, a model that thinks for 500 tokens before a 150-token answer more than quadruples the output side of the bill. For routine questions like “what are your opening hours”, lower the reasoning setting or use a model that does not reason. OpenAI’s max_output_tokens parameter caps reasoning and visible output together, though a cap set too low can return an incomplete response with no visible answer.
Caching: the biggest lever, with conditions
Prompt caching stores the start of a request so the next request that begins the same way reads it at a discount. On the prices above, a cached read costs a tenth of normal input. In our example it cuts the Claude bill by about 64% and the OpenAI bill by about 62%.
Those savings depend on conditions the headline rate does not show:
- Writing costs extra. Anthropic charges 1.25 times the input price to write to its 5-minute cache, and OpenAI’s price list has its own cache-write column. Our figures include those charges. Leaving them out would overstate the savings.
- Short prompts do not cache. The minimum is 1,024 tokens for Claude Sonnet 5 (Anthropic) and for GPT-5.6 models (OpenAI). A 300-token system prompt gets no discount.
- The start has to be identical. Keep fixed instructions first and anything that changes, such as the date or the customer’s name, after them. One changed character near the top breaks the match.
- Caches expire. Our table assumes every turn hits the cache. With Claude’s 5-minute cache, a customer who takes longer than that to reply turns the next request into a fresh write.
- Setup differs. OpenAI enables caching by default on supported models. On Claude you opt in, either by marking cache points or with a single automatic setting suited to multi-turn chats.
A cheaper token is not always a cheaper bill
Comparing vendors by price per token assumes a token is the same unit everywhere. It is not. Each vendor splits text with its own tokenizer, so the same message can be a different number of tokens. Anthropic’s pricing page notes that its newer tokenizer, used from Claude 4.7 onward, produces approximately 30% more tokens for the same text than its previous one.
Our table uses identical token counts for all three models to keep the arithmetic readable. Before you choose a model on price, send the same real conversation to each candidate and compare the token counts reported back, not only the rate card.
Other things that change the rate
- Batch processing halves the price at all three vendors for work that does not need an instant answer, such as overnight summaries or tagging a backlog.
- Long prompts can cost more per token. OpenAI lists separate long-context rates; for GPT-5.6 Terra they are $4 input and $18 output. Google charges more for Gemini 3.1 Pro prompts over 200,000 tokens. Anthropic says Claude 4.6 and later models bill the full 1-million-token window at the standard rate.
- Promotions end. Besides the Gemini date above, OpenAI says GPT-5.6 Sol’s promotional pricing runs at least through 21 November 2026.
- Agents multiply everything. An agent sends its growing record of every step back with each call, so a 20-step task is 20 requests with a longer history each time. That is one reason how AI agents work matters for your budget as much as for reliability.
Reading your own bill: a checklist
Every API response reports token usage. Log it, then check these in order:
- Input vs output split. If input dominates, as in our example, history and instructions are the problem. Trim the instructions, and cap how many past messages you send or summarise older ones.
- Cache hits. On OpenAI, compare
input_tokens_details.cached_tokenswith total input tokens. On Claude, look atcache_read_input_tokensagainstcache_creation_input_tokens. Lots of writes and few reads means your prompt start is changing or the cache is expiring. - Hidden reasoning. OpenAI reports
output_tokens_details.reasoning_tokens; Claude reportsoutput_tokens_details.thinking_tokens. Compare them with the length of the visible replies. - Tool and search overhead. Tool definitions are sent as input on every request, and some tools are priced separately; Anthropic charges $10 per 1,000 web searches on top of tokens.
- Model fit. Try the same logged conversations on a smaller model. Tighter instructions help here too, and our practical guide to prompting covers writing them.
The thing worth remembering
You are billed for what the model reads, every time it reads it. Estimate from the full request, including history and hidden reasoning, and make the fixed start of every request identical so caching can do its work.
Header photo: N509FZ / Wikimedia Commons.jpg), CC BY-SA 4.0.