Businesses budget for building an AI feature and are surprised by what it costs to run. Unlike ordinary software, where hosting is roughly fixed, AI cost scales with use, and it can scale in ways nobody modelled.
Here is how the bill is actually constructed and what makes it grow.
How the pricing works
Language model providers charge per token, where a token is roughly three quarters of a word. You pay for both what you send and what comes back, and output is typically several times more expensive than input.
The important consequence: you pay for everything you send, every time. If each request includes a long system prompt, ten examples and five retrieved documents, you are paying for all of that on every single call, whether or not it was needed.
A worked example
A customer support assistant answering questions from your documentation.
Per conversation, a realistic shape:
- System instructions: around 500 tokens.
- Retrieved documents to ground the answer: around 2,000 tokens.
- Conversation history by the third message: around 800 tokens.
- The user’s question: around 100 tokens.
- The answer: around 300 tokens.
That is roughly 3,400 input tokens and 300 output tokens per exchange, and a typical conversation has three or four exchanges.
At a few thousand conversations a month, the model cost usually lands somewhere in the tens of thousands of rupees. At ten times that volume it is a line item that needs managing rather than a rounding error.
The point is not the exact number, which moves as providers change pricing. The point is the shape: cost is driven by how much context you send, multiplied by how often.
The three things that multiply the bill
1. Sending more context than the question needs
The most common cause. A system that retrieves ten documents when two would do is paying five times more on every request, forever.
Better retrieval is a direct cost reduction, not just a quality improvement.
2. Conversation history growing without limit
If every message includes the entire conversation so far, a long conversation costs dramatically more at message twenty than at message two, because the same text is resent each time.
Summarising older history, or keeping only the last few exchanges, keeps this flat.
3. Using the largest model for everything
Most requests in a typical product do not need the most capable model. Classifying an incoming message, extracting a field, or answering a simple factual question can run on a smaller and much cheaper model.
Routing by difficulty, with the large model reserved for the hard cases, commonly reduces cost substantially without a noticeable quality change.
The costs beyond the model
- Vector database or search infrastructure for retrieval, if you have one.
- Ordinary hosting for the application around the model.
- Embedding generation when documents are added or updated. Small per document, and it recurs whenever you reindex.
- Monitoring and logging. Storing prompts and responses for quality review, which you should do, has a storage cost.
- Human review time, especially in the first months. Not a software cost and frequently the largest one.
How to keep it predictable
- Cache repeated answers. If the top twenty questions are forty percent of volume, answering them from a cache removes forty percent of the cost.
- Retrieve less, better. Two well chosen documents beat ten mediocre ones on both quality and price.
- Trim conversation history with a fixed window or a running summary.
- Route by difficulty. Small model first, escalate only when needed.
- Set hard spend limits at the provider. Every provider supports this and it is the protection against a loop or an abusive user producing a bill nobody expected.
- Rate limit per user. Prevents both accidents and deliberate abuse of a public facing feature.
- Log cost per feature, not just in total, so you know which feature is expensive and whether it is worth it.
The question worth asking before building
What is the cost per useful outcome, compared with the alternative?
For a support assistant: cost per conversation resolved, against the cost of an agent handling it. For document processing: cost per document, against the minutes of typing it replaces. For a sales tool: cost per proposal drafted, against the hours saved.
If a feature costs more per outcome than the human process it replaces, it is a product decision rather than an efficiency one, and it should be justified on that basis. Our notes on when AI support is worth it and document automation payback work through both calculations.
What to expect by feature type
- Internal assistant, tens of users: usually a modest monthly cost, dominated by hosting rather than model usage.
- Customer support automation, thousands of conversations: a real line item, controllable with caching and routing.
- Document processing at volume: priced per page, and generally the easiest to forecast because volume is known.
- Public facing consumer feature: the one that needs hard limits, because usage is not under your control.
Frequently asked questions
How much does it cost to run an AI chatbot?
It depends on conversation volume and how much context each request carries. The controllable factors are retrieval size, history length and model choice, and together they can change the bill several fold.
Why did our AI bill increase without more users?
Usually because conversations got longer, retrieval was widened, or the prompt grew. All three increase tokens per request while usage looks unchanged.
Can we run models ourselves to avoid per token costs?
Self hosting trades usage cost for fixed infrastructure and engineering time. It becomes worthwhile at high steady volume and is usually a poor trade below that.
How do we avoid a surprise bill?
Set hard spend limits at the provider, rate limit per user, and alert on daily spend. Do this before launch, not after the first incident.
Ezitech builds AI features with cost controls designed in, and will tell you when the economics do not work. See our AI solutions or describe what you want to automate.
Related: What software actually costs in Pakistan — every Ezitech cost guide in one place.