A router buys you one integration, one bill, and the ability to change models without shipping code. Calling direct buys you the provider’s newest features on the day they ship and one less party between you and the model. The decision is about how many models you actually run, and the real risk is not price — it is silent behaviour change on a swap.
What the router actually buys
One integration instead of several. The obvious one, and it matters more than it sounds when a product uses different models for different jobs — a cheap one for classification, a strong one for the hard step.
Model choice as configuration. This is the one I value most. Changing which model backs a feature becomes a value in the database rather than a deploy. That turns a model swap from an engineering task into an operational one.
A fallback path. When a provider has a bad hour, you can route around it. For anything user-facing that is worth real money.
One bill. Administrative rather than technical, and still genuinely useful.
What direct buys
New features immediately. Providers ship capabilities that take time to appear through an intermediary. If you need the newest tool-calling or structured-output behaviour, direct is where it lands first.
One less party. Fewer moving parts, fewer places for a failure to originate, and no margin.
Provider-specific behaviour you can rely on. If you have tuned prompts against one model’s quirks, the abstraction is not buying you portability anyway.
Side by side
| Router | Direct | |
|---|---|---|
| Integrations to maintain | One | One per provider |
| Switching models | Config change | Code change |
| Fallback on outage | Available | Build it yourself |
| Newest provider features | Lags | Immediate |
| Cost | Margin on top | Provider price |
| Failure surface | Two parties | One |
| Best when | Several models, want flexibility | One model, want depth |
The failure nobody warns you about
A model swap is not a configuration change, even when it looks like one.
Models differ in how they handle tool calling, how strictly they honour structured output, how they behave with long context, and how they respond to the same prompt. Switching the string in a config field can quietly change what your product does.
Two specific things I have hit:
Optional string fields arriving as empty strings. Some models send an empty string where you expected the field to be absent, and if your validation treats empty as invalid, the call fails in a way that looks like a model problem rather than a serialisation one. Strip empty strings before validation.
Reasoning behaviour on by default. A model that reasons before answering can consume its entire output budget thinking and return an empty result with a successful status code. A 200 with no content is the most confusing possible failure, because nothing errored. If a model exposes a reasoning setting, set it explicitly rather than accepting the default.
Both of these were invisible until a real product depended on them, and neither is the router’s fault. They are what “portability” actually means in practice: the plumbing is portable, the behaviour is not.
The operational risk
A drained prepaid balance looks exactly like an outage.
With a router, credit is usually prepaid. When it runs out, calls fail, the product stops working, and nothing in your logs says “you ran out of money.” I have had this happen on more than one account.
Alert on balance the way you would alert on disk space. It is the same class of problem: a resource that depletes silently and takes the service down when it hits zero.
The related trap: do not track spend by summing what each call reports. Trust the balance endpoint. Reported per-call charges and the actual balance movement do not always agree, especially if the key is shared across several of your own tools.
How I split it
- One model in production, no plans to change? Direct.
- Several models for different jobs? Router, for the single integration alone.
- Need to switch without shipping? Router.
- High volume, cost-sensitive? Direct, and build your own fallback.
- Either way: keep the model choice in configuration, not scattered through the code. That is what makes the decision reversible.
Practical notes
- Test every model swap against real prompts, especially anything using tools or structured output.
- Strip empty strings before validating model output.
- Set reasoning behaviour explicitly where the option exists.
- Alert on credit balance.
- Log which model actually served each request. When quality changes, the first question is what answered, and you cannot reconstruct that later.
FAQ
Router or direct?
Router for several models, config-level switching, and fallback. Direct for one model and the newest provider features.
Does a router cost more?
There is a margin, against one integration and one bill. At high volume the margin dominates.
Biggest risk?
Silent behaviour change on a swap. Models differ on tools, structured output and long context. Test, never assume.
Can you run out of credit mid-production?
Yes, and it looks like an outage. Alert on balance like disk space.
Related: Claude vs GPT for coding, MCP vs function calling, and AI agent development.