Tokens Are the Budget: How Free LLMs Forced Us to Optimise Our MCP Tools

When we brought Faydo — a marketplace for discounted gift cards — to WhatsApp, we made a deliberate choice: build it on free and low-cost language models first. Not because we couldn’t afford a premium model, but because WhatsApp is a volume channel. If every incoming message triggers an expensive LLM call, the economics only work at massive margins. Prove it can run lean, and it can run anywhere.

So we wired up Gemini 2.5 Flash (on Google’s free tier) as the primary model, with Groq as a fast fallback. Then reality hit: free tiers come with tight rate limits and token quotas. We started seeing 429 Too Many Requests. Long tool outputs blew through context budgets. A chatty design that felt fine in testing fell over the moment traffic arrived.

That constraint was a gift. It forced a discipline we’d recommend to anyone building agentic commerce: treat tokens as your budget, and spend them like they’re real money. Here’s what changed.

The biggest hog: re-sending the whole toolbox every turn

The default way to let a model use tools is “native function-calling” — you hand the model the full JSON schema of every tool on every single turn, and it replies with a structured call. It’s elegant. It’s also expensive. Our tool catalogue is roughly 3,000 tokens, and native function-calling re-sends all of it, every message, forever — even for “hi” or “show me more.”

On a free model with a tight budget, that’s the difference between working and rate-limited.

Our fix was to stop treating the toolbox as something the model re-reads constantly. Instead, we put a terse action list in the system prompt — one short line per action — and ask the model to reply with either plain text or a single compact JSON action:

Instead of a 3,000-token tool schema every turn, the model sees:

  brands(query)         → search the catalogue
  brand(slug)           → a brand's discount, prices, redemption
  buy(slug, amount)     → create an order + UPI link
  ...

And it answers with ONE line:

  {"act":"brand","slug":"zomato"}

The gateway runs that single MCP call and hands back the
already-formatted result. ~1 model call per turn, ~90% fewer
tokens per call.

Same outcome — the model still drives the tools — at a fraction of the token cost. This one change did more for our bill and our reliability than anything else.

Show the model only the tools that fit the intent

Even a compact action list is worth trimming further. Almost every message has one obvious intent — browsing, buying, or checking an order — and each intent only needs a handful of tools. Forcing the model to weigh every possible action on every turn is wasted reading (tokens), slower decisions, and a fresh chance to reach for the wrong tool.

So before the model reasons about anything, a lightweight check reads the intent — from the words themselves (“buy”, “₹1000”, a tapped button) and from where the user is in the conversation — and narrows the toolset accordingly:

• A clear purchase intent goes straight down a focused purchase path, instead of asking the model to rediscover how to buy from first principles.
• Account actions — your vouchers, your past orders — only appear once you actually have an account. There’s no reason to show them to a first-time browser.
• The “set up your profile” action surfaces only at the exact moment it’s needed, right before a first purchase — and disappears again afterwards.

The effect is that the model always reasons over the right small set of tools for what you’re actually doing — never the full catalogue. Fewer tokens to read, quicker decisions, and far less chance of picking the wrong tool. It’s the same instinct a good shop assistant has: don’t recite the entire rulebook to someone who just asked where the fashion cards are.

Ground the model locally so it doesn’t “call to find out”

A subtle token drain is the model making a tool call just to figure out what it’s looking at. “Do you have Tata CliQ?” shouldn’t require a round-trip if we can answer it instantly.

So the gateway keeps a small in-memory catalogue of every brand — names and slugs, refreshed periodically from our API. Before any tool call, it resolves what the user meant locally: “Tata CliQ,” “tata-cliq,” and “tatacliq” all collapse to the same brand. This does two things at once — it saves a tool round-trip, and it grounds the model so it can only ever fetch or buy a brand that genuinely exists on Faydo. Fewer tokens and fewer hallucinations.

Make each tool earn its round-trip: composite tools

Every tool call is a full model turn — context in, decision out. So the fewer turns a task needs, the cheaper it is. A naive purchase flow is four steps: price it, start payment, check payment, deliver the card. That’s four model turns.

We collapsed them into two composite tools:

• start_purchase does the pricing and returns the UPI payment link in one call.
• complete_purchase verifies the payment and delivers the gift card in one call.

Two turns instead of four, for the exact same purchase — and the guardrails (validating the amount, confirming before charging) still live safely on the server.

Trim what comes back

Tokens are spent on the way out of a tool too. A brand’s full terms and redemption steps can be pages of rich text; dumping all of it into the model is wasteful. So tool outputs are deliberately compact: denomination lists are capped (with “…more on the brand page”), redemption steps are trimmed to a readable snippet, HTML is flattened to plain text, and a tiny machine-readable footer carries the few exact values the model needs to drive the next step. The model gets enough to answer accurately, and not a token more.

Route the easy stuff around the model entirely

Here’s the mindset shift the free tiers really taught us: not every message deserves an LLM call. “1”, “yes”, “show more”, tapping a menu option — these are unambiguous. Spending a model turn to interpret them is pure waste.

So the gateway runs in a hybrid mode. Clear, structured input is handled by a plain deterministic flow — zero tokens, instant, perfectly predictable. The language model is reserved for what it’s actually good at and what actually needs it: interpreting genuine, fuzzy natural language. The cheapest LLM call is the one you never make.

On multi-step agents and “sub-agents”: fewer, sharper turns win

It’s fashionable to solve hard tasks by spinning up elaborate agent loops — a planner, a router, several specialised sub-agents, each taking its own turn. It can be powerful. It is also a token multiplier: every extra turn re-pays the cost of the context, and on a rate-limited free model those turns stack up into 429s fast.

Our constraint pushed us the other way. Instead of fanning out to a swarm of specialised sub-agents — each rediscovering context and paying for its own turn — we do the equivalent work with intent routing inside a single turn: read the intent cheaply, point the model at just the toolset that fits it, and do the heavy lifting deterministically in code where no model is needed at all. When we do need a second model pass — say, to phrase a tool’s result nicely for WhatsApp — we make it count and keep it minimal. One sharp, well-scoped turn beats a fleet of thin ones, especially when you’re paying by the token.

Graceful under pressure

Finally, we planned for the limit instead of pretending it wouldn’t come. When the primary model returns a 429, the gateway automatically falls back to the secondary provider rather than failing the conversation. A rate limit becomes a slightly different model, not a dead end.

A boundary worth respecting: tool design vs. model orchestration

There’s a question that naturally follows all of this: if these optimisations are so valuable, why aren’t they all inside the MCP server itself — the one place every channel shares?

Because they aren’t all the same kind of optimisation. There are two layers, and they belong in two different places.

Layer one is the tools themselves. Making tools coarser (one start_purchase instead of four steps), trimming what they return, writing descriptions that keep the model honest, pricing from a single source of truth — these make the tools better, so they help every client equally: ChatGPT, Claude, our own app, and the WhatsApp gateway. These live in the MCP server, and they should.

Layer two is how a given client drives its own model. The compact action format, the intent routing, the provider fallback, the decision to skip the model entirely on unambiguous input — these are about talking to one particular language model efficiently. And here’s the catch: the MCP server has no model. It’s a stateless tool provider. The clients are the ones that run an LLM. ChatGPT and Claude use their own native tool-calling and manage their own token budgets; our gateway runs a free model and needed its own tricks to survive on it. You can’t push that logic into the server — and you shouldn’t, because it would weld a clean, shared, standard interface to one client’s private model choices, without helping the clients that don’t use it.

The rule of thumb we landed on is simple:

If an optimisation makes the tools better, it belongs in the MCP server and everyone inherits it. If it’s about talking to a particular model, it belongs in the client.

Keeping that line sharp is what lets one boring, standard MCP server sit behind five very different front doors. And when several of your own clients happen to share the same cheap model — say a WhatsApp gateway and a future web chat — the home for that shared orchestration is a small agent library that sits between your clients and the server, not the server itself. The server stays LLM-agnostic; your clients share the model-specific cleverness. Everyone keeps their job.

The takeaway

Building on free LLMs sounds like a limitation. In practice it was a forcing function for good engineering. Every optimisation above — compact tool-calling, local grounding, composite tools, trimmed outputs, deterministic routing, restrained agent loops — makes the system cheaper, faster, and more reliable on any model, free or premium. The constraint didn’t hold us back; it made the product lean enough to run anywhere our customers are.

If you’re building agentic commerce, start by pretending your model is free and rate-limited. You’ll design something better than if you’d assumed infinite tokens.