Skip to main content
OneAdvanced Software (return to the home page)

AI token optimisation: How organisations can get more value from every AI token

Learn how enterprises can optimise AI token usage with model routing, caching, prompt design, budgets and FinOps, cutting costs without losing output quality.

by Swet Kamal Content Executive

Published on 17 November 2026 8 minute read

Key takeaways

  • •Cheaper tokens don’t mean cheaper AI. Per-token prices are falling, but agentic workflows can use 5 to 30 times more tokens per task than chatbots, so total costs can still climb sharply.
  • •Optimise for value, not volume. Track cost per outcome (such as cost per resolved case) rather than total tokens consumed. A pricier model can be the better investment if the outcome is worth far more.
  • •Waste is widespread. Respondents in Harness’s survey estimated 26% of AI spend is wasted, mostly through verbose prompts, oversized models, bloated context and runaway agent loops.
  • •Match the model and the context to the task. Route simple work to smaller models, cache repeated prompts, retrieve only relevant information and cap output length.
  • •Control agents and build visibility. Set step budgets, timeouts and retry limits, and tag every model call by team and use case so costs have a clear owner.
  • •Measure first, then optimise continuously. A 90-day roadmap (baseline, route and cache, then govern) helps you cut costs without sacrificing quality, and regular reviews keep the savings in place as models and pricing change.

AI tokens are getting cheaper, with Gartner predicting the cost of running inference on a trillion-parameter model to fall by more than 90% by 2030. Yet the cost of running each agentic workflow could rise more than fivefold by 2028.

At OneAdvanced, we see this challenge first-hand. Organisations we serve across healthcare, legal, government, and social care face a common question: how can they scale AI without losing control of costs? The answer is AI token optimisation, the discipline of getting more value from every token, enabling organisations to scale AI and keep costs under control

What is AI token optimisation?

AI token optimisation is the practice of reducing the tokens an AI system uses per task through model selection, prompt design, caching and context management, without compromising result quality. Done well, it reduces the costs per useful outcome.

So, what are tokens? A token is a small unit of text that an AI model processes. According to the FinOps Foundation, roughly 1,500 words equate to 2,048 tokens. AI providers usually charge for three types of tokens:

  • Input tokens: The prompts, instructions, documents and conversation history sent to the model.
  • Output tokens: The text generated in response.
  • Reasoning tokens: The additional tokens some models use for internal reasoning before producing an answer.

Each type may carry a different cost, depending on the model and provider. A model that consumes ten times more tokens but delivers an outcome worth a hundred times more could well be the better investment. Hence, organisations should look beyond the cost of induvial tokens to the value AI delivers.

Why enterprise AI costs compound so quickly

AI providers typically charge per million input and output tokens. While the cost of an individual request may be small, consumption can rise quickly when AI becomes part of wider business workflows. There are two prime factors contributing to this.

1. Agents multiply tokens consumption

Unlike standard chatbots, AI agents break tasks into steps, call tools, review results and retry when needed. Gartner's March 2026 analysis estimates that agentic AI models can use 5 to 30 times more tokens per task than standard chatbots.

Each call made by an agent may include system instructions, conversation history and retrieved documents, adding to token consumption. For example, sending a 10,000-token prompt across 50 calls consumes 500,000 input tokens alone. Multiply this across users and workflows, and costs can quickly escalate.

2. RAG sends more text than necessary

Retrieval-Augmented Generation (RAG) helps AI generate relevant answers by retrieving information from documents and databases. However, retrieving too much information, passing entire document extracts instead of relevant passages, or repeatedly sending unchanged context can waste tokens and simultaneously will inflate costs.

By retrieving only the information needed, removing redundant content and avoiding unnecessary repetition, organisations can reduce token consumption without compromising answer quality.

Where tokens get wasted

In Harness’s 2026 survey of 700 engineering leaders, including 100 in the UK, respondents estimated that 26% of AI spending is wasted. This waste typically comes from four areas:

Waste source

What it looks like

How to reduce it

Verbose system prompts and repeated context

Sending lengthy instructions and policy documents to the AI model every time it is used.

Shorten instructions, use templates and reuse information where possible.

Oversized models for simple tasks

Using advanced AI models to classify emails or answer frequently asked questions when simpler models could do the job.

Use the correct model for the task.

Unfiltered retrieval and bloated context windows

Providing entire documents when only a paragraph or a few details are needed.

Retrieve less, rank better andprune before sending

Verbose outputs, retries and runaway loops

Generating lengthy answers, retrying failed requests too often or allowing AI agents to continue working without clear limits.

Cap the outputs, limit retries and set step budgets

Proven token optimisation techniques

AI token optimisation isn’t about using AI less. It’s about using the model that most suits the task, sending only relevant information and avoiding unnecessary processing. The following eight techniques can help organisations reduce token consumption while maintaining performance and accuracy.

1. Model routing and tiering

Not every business task needs to use most powerful AI model. Using a frontier model for routine requests can increase costs without delivering meaningful results. A tiered approach routes simpler tasks to smaller, more economical models and reserves more capable models for work that requires deeper reasoning, greater accuracy or complex decision making.

2. Prompt caching

AI applications often send the same instructions, policies or reference material with multiple requests. Prompt caching allows LLM models to reuse previously processed portions of a prompt instead of reprocessing the same content. This can reduce input-token costs and improve response times, particularly in applications with large, frequently repeated prompts.

3. Prompt design and compression

System prompts can grow over time as teams add instructions, examples, and background information. Removing repetition and simplifying lengthy explanations can reduce token consumption across repeated requests without affecting output quality.

For example, reducing a 4,000-token prompt to 2,200 tokens saves 1,800 input tokens per request. Microsoft's LLMLingua research also demonstrated the potential of prompt compression, reducing a 2,366-token prompt to just 117 tokens, with an approximately 1.5-point performance drop on reasoning benchmarks.

4. Context pruning and smarter retrieval (RAG)

Sending entire documents, lengthy conversation histories or irrelevant background information to an AI model wastes tokens and can obscure relevant details. Retrieval-augmented generation (RAG) addresses this by retrieving relevant information from a knowledge base and providing it to the model. Context pruning goes a step further by removing duplicate, outdated or unnecessary information before it reaches the model.

5. Output length and format control

Longer AI responses consume more output tokens and increase costs. Setting clear response length limits, specifying the required format and matching the level of detail to the task can prevent unnecessary token use.

Format matters, too. According to the FinOps Foundation, CSV can use 30-60% fewer tokens than JSON for tabular data, making simpler formats a practical way to reduce consumption where appropriate.

6. Batching for non-urgent work

Not every AI task needs an immediate response. Batching allows organisations to process multiple requests together, making it useful for high-volume, repetitive tasks that can be handled on a schedule. For example, a retailer could analyse thousands of customer feedback responses in a scheduled batch rather than process each response as it arrives. This helps identify recurring complaints and product issues while potentially reducing processing costs and request handling overhead.

7. Semantic caching

Users often ask the same question in different ways. Semantic caching identifies queries with similar meanings and reuses a previously generated response instead of making another model call. This reduces processing costs and improves response time, particularly for high-volume repetitive queries.

For example, an employee might ask, "How do I reset my password?" or "I've forgotten my login password. What should I do?" An internal AI assistant can recognise the shared intent and return the same approved answer. Cached responses should still be checked for relevance, accuracy and freshness, particularly when information changes frequently or depends on the user's circumstances.

8. Agent design

AI agents can consume excessive tokens when they repeat tasks, make unnecessary tool calls, or continue working even after completing a task. Well designed agents use clear instructions, limit the number of steps and retries, and invoke tools only when necessary. This keeps costs under control without compromising results.

Optimising agentic AI workloads

AI agents can be a major driver of token consumption. They work across multiple steps, call external tools, review results and retry failed tasks, with each step potentially triggering additional model calls. As organisations move from AI assistants to agentic workflows, these costs can quickly multiply.

The solution isn’t to limit what agents can do, but to make them more efficient.

  • Set step budgets to control how many actions an agent can take.
  • Use timeouts to prevent processes from running indefinitely.
  • Limit retries to avoid repeated, costly attempts at failed tasks.
  • Introduce human checkpoints for high-impact decisions, allowing agents to escalate issues rather than continue without reliable results.

Agent design matters here. Specialised agents built for specific tasks can reduce unnecessary processing compared with general purpose agents operating without clear boundaries. In OneAdvanced IQ, agents are embedded in workflows to automate key processes rather than operate as standalone tools. Our AI agents marketplace offers specialised agents, while intelligent chat agents provide teams with a single, governed access point.

For practical guidance, read our guide to [agentic AI implementation] [Add link].

Build token visibility: Measurement and FinOps

You cannot optimise what you cannot see. Effective token optimisation starts with visibility, accountability and continuous monitoring.

  • Measure business value, not just token volume: Track tokens per task, cost per outcome (such as cost per resolved case), cache hit rates and latency.
  • Attribute costs and set budgets: Tag every model call by team, use case and model to establish clear cost ownership. Start with showback to make spending visible, then introduce chargeback once cost allocation is reliable. Set token budgets for each feature or agent, with alerts to flag potential overspending.
  • Monitor spending regularly: A Harness survey found that 72% of respondents had received unexpected AI bill in the past year, while 52% said AI costs lacked clear ownership. A further 42% reviewed AI costs only quarterly, making it harder to identify and address overspending quickly.
  • Enforce cost policy: Policies only work when enforced. Harness found that while 73% of respondents had AI cost policies, only 47% fully enforced them. Organisations need clear ownership, automated controls and regular reviews to turn cost policies into effective action.

API, private or sovereign Hosting? The economics

Running your own AI model can be cheaper than using a premium API, but only at sustained volume. Researchers at Carnegie Mellon University modelled 54 deployment scenarios, found that smaller open models running on hardware costs about 2,000 pounds could pay back in three months. Mid sized models take roughly 4 to 34 months, depending on the API they replace.

The right approach depends on your workload, budget and data requirements.

Option

Control

Cost predictability

Data residency

Flexibility

Best for

Public API

Low to medium

Variable; scales with usage

Depends on provider and region

High; switch models easily

Pilots and low or spiky volumes

Private (self-managed)

High

Predictable if well utilised; idle capacity costs

You choose the region

Medium; you run the stack

Steady, high-volume workloads

Steady, high-volume workloads

High

Capacity-based; fewer metering surprises

UK-only processing

Medium; curated models and agents

Regulated sectors needing UK residency

However, cost is only one constraint. Healthcare, legal, and public sector organisations must also consider data protection, data residency, and procurement requirements when choosing an AI deployment model. Our AI governance framework guide explores the controls needed to adopt AI responsibly.

Cost can also become a barrier when it comes to scaling AI. Research from SAS found that 41% of UK and Irish enterprises that had scaled generative AI considered LLM costs prohibitive. This is where OneAdvanced AI fits. Embedded into OneAdvanced IQ, our agentic systems are designed to be trusted as well as intelligent. They use a sovereign LLM developed in partnership and hosted exclusively in the UK, alongside self-hosted models and retrieval pipelines that keep the context supplied to AI focused and relevant. With more than 50 purpose-built agents embedded in the flow of work, organisations can adopt specialised AI without building and maintaining every agent themselves.

A 90-day token optimisation roadmap

Token optimisation rarely fails because the techniques don’t work. It fails when teams implement changes without a baseline, ownership or plan. This 90-day roadmap help organisations identify savings, optimise AI workloads and maintain costs under control as adoption grows.

Days 0-30: Establish your baseline and capture quick wins

  • Identify AI workloads and assign an owner to each.
  • Track token usage and costs by team, use case and task.
  • Reduce unnecessary token consumption by shortening prompts, limiting output length and batching non urgent tasks.

Days 31-60: Optimise model routing and processing

  • Route tasks to the most cost effective model that meets your quality requirements.
  • Enable prompt caching for highest-volume workloads to reduce repeated processing.
  • Remove irrelevant information from retrieved data and validate output quality after each change.

Days 61-90: Embed governance and ongoing control

  • Set spending budgets and alerts for each AI feature or agent.
  • Limit agent steps to prevent unnecessary processing and token consumption.
  • Publish monthly cost reports to give teams visibility of AI spending.
  • Review token usage and costs whenever models, workloads or pricing change.

Five mistakes that undermine token optimisation

  • Optimising before measuring: Without a baseline, you cannot measure savings or demonstrate the value of token optimisation to your CFO.
  • Cutting quality to reduce costs: Cheaper AI is not better if inaccurate answers lead to rework, delays or poor decisions.
  • Using one model for every task: Different tasks need different levels of AI capability. Using a powerful model for routine tasks can increase costs unnecessarily.
  • Ignoring agent loops: Agents that repeatedly retry tasks or make unnecessary model calls can quickly drive up token consumption and erase savings.
  • Treating optimisation as a one-off exercise: Model prices, capabilities and workloads change over time. Regular reviews help ensure you continue using the most cost effective model for each task.

Conclusion: Make every token count

As AI adoption grows, controlling token costs will become increasingly critical. Organisations need to monitor usage, choose the right models, reuse existing results and ensure every AI task delivers value. OneAdvanced IQ brings people, data, and AI together in a connected, trusted, and intelligent system of work.

Frequently Asked Questions

How are AI tokens priced?

Most providers charge per million tokens, with separate rates for input and output. Output usually costs several times more than input. Reasoning tokens are typically billed like output. Rates and billing models change often, so check current terms before budgeting.

Why do AI agents use so many tokens?

An agent works in a loop. It plans, calls tools, reads results and checks its own work, and every step re-sends the instructions, and everything gathered so far and all are done using tokens.

How do I estimate or forecast AI token costs?

Start with the basics: tokens per task, multiplied by tasks per day, multiplied by your price per token. Measure tokens per task in a pilot rather than guessing. For agents, forecast a range instead of a single figure, because loop depth varies. Add a buffer for retries and growth and review the forecast monthly against actual usage.

How do I measure ROI per token?

Measure cost per outcome rather than cost per call, such as cost per resolved case or per document processed. Then compare it with what that outcome is worth, in time saved or revenue protected. Total tokens consumed is an activity metric, not a value metric.

How does UK data sovereignty affect AI costs?

UK residency requirements narrow your options. Many public AI APIs process data outside the UK or give you limited control over where it goes. Regulated sectors such as healthcare, legal and the public sector may therefore need private or UK-hosted deployment, which changes the cost model from pay-per-token to capacity-based. The extra cost is often cheaper than the compliance risk it removes.

About the author


Swet Kamal

Content Executive

Since joining OneAdvanced in June 2023, Swet has become a trusted voice on how businesses can make sense of technological change. As a Content Executive, she crafts thought leadership, SEO blogs, PR content and guides that turn complex developments into clear, practical perspectives for UK business audiences. Her expertise spans AI, digital transformation, cybersecurity and workflow automation. Bridging the gap between insight and action, she helps decision-makers navigate change with confidence and stays ahead of emerging trends through continuous learning.

Share

Contact our sales and support teams. We're here to help.

Contact Us
Speak to our sales team

Speak to our expert consultants for personalised advice and recommendations or to book a demo.

Call us on

0330 343 4000