Model Tiering: Why I Run 3 AI Models Instead of 1 (And Save 70%)
I used to fire every prompt at the same model. Same API, same per-token rate, same wait. Then I looked at the bill and realized I was paying GPT-4 prices just to figure out if an email was spam.
That’s what pushed me to start tiering. Three models now, each doing the job it’s best at, and I’m spending about 70% less. There’s more wiring on my side to make it work, but I don’t mind.
What Model Tiering Actually Means
It’s a simple idea. You stop sending every prompt to the biggest model you can find. Easy tasks go to a small model. Hard tasks go to the big one. You match the tool to the job instead of paying Ferrari prices to grab groceries.
This is how good engineers have always thought about compute. Run cheap workloads on cheap hardware. Save the expensive gear for what actually needs it.
Most of what I send to AI day-to-day isn’t that hard. Stuff like support ticket triage or short doc summaries. A small model running locally on Ollama handles all of that, and it costs me basically nothing to run. That’s not work that needs a frontier model. And it shouldn’t cost what it does if you’re piping everything through one API endpoint.
Here’s what I’m actually running right now. Three tiers, each with a specific job.
Tier 1: Local Ollama (Llama 3.1 8B). Runs on a box in my closet. Zero per-token cost. I use it for classification, short rewrites, regex-style extraction. Anything where the prompt fits in a small context and the answer fits in one paragraph. Latency is fine. Quality is fine for what it does. The tradeoff is real though: it hallucinates more than the paid models, so I don’t trust it for anything that goes out without me reading it first.
Tier 2: Claude Haiku (or GPT-4o-mini depending on the day). This is my workhorse. Code reviews, longer summaries, draft emails, anything that needs more nuance but isn’t going to break the bank. Cheap. Fast. Good enough 90% of the time. Sometimes it’s just wrong in a confident-sounding way, which is why I still skim the output before I paste it anywhere.
Tier 3: Claude Sonnet (or whatever the current “smart” model is). Reserved for the hard stuff. Architecture decisions. Long-form content that needs to actually be good. Code that’s going into production. Maybe 10 to 15% of my traffic lands here, and that’s the point. If I sent everything through Sonnet I’d be burning cash on prompts a $0.0001 model could handle.
Where the 70% Comes From
The pricing does most of the talking. Run the numbers and it’s not subtle:
| Task Type | Before (single model) | After (tiered) |
|---|---|---|
| Classification & extraction | Sonnet ($3 / $15 per 1M tokens) | Ollama 8B (free) |
| Drafts & rewrites | Sonnet | Haiku ($0.25 / $1.25 per 1M tokens) |
| Architecture & hard reasoning | Sonnet | Sonnet (no change) |
Push 60% of calls from Sonnet down to Haiku, move another 25% to a free local model, and your bill drops fast. That’s where the 70% comes from. Most folks running tiered setups land somewhere in the 60-75% range on workloads like this.
I run Ollama locally for the classification and extraction work, so that side I know firsthand. For the cloud tiers I’m going off published rates and what people are reporting in community threads. Your mileage will depend on what you’re actually pushing through. If half your calls are hard reasoning, the savings shrink. If most of it is summarization and extraction, you’ll see something close to the table above.
The Trade-offs Nobody Talks About
It’s not free money. There are real costs.
Routing logic isn’t trivial. You need something to decide which model gets which task. A script, a Make scenario, an n8n flow. Mine’s a basic difficulty classifier at the front of the pipeline. Prompt under 500 tokens asking for a yes/no? Local model. Anything else? Send to the API. It works most of the time. But it misroutes occasionally and I have to fix it.
Latency varies by tier. Local Ollama is fast until it isn’t. If the box is busy doing something else, your “free” tier suddenly takes 8 seconds. Plan for it.
Quality isn’t uniform across models. From what I’ve seen in vendor docs and community threads, the smaller tiers handle simple tasks fine but stumble on anything requiring real reasoning. People report Haiku hallucinating things like nonexistent Python libraries. Sonnet apparently doesn’t do that as often, though I’d want to verify it myself before betting a production pipeline on it. You give up something on the middle tier. Know where that line sits for your workload.
How I Set It Up
You don’t need to tear anything down to try this.
- Pull a week of API logs. Sort calls by how complex they actually are.
- Flag the 40 to 60% that a small model can clearly handle.
- Route those to Haiku or a local Ollama box.
- Monitor quality for a week. Bounce anything back that breaks.
- Run it again next month.
That’s the whole thing. No new platform, no new vendor. Just cheaper inference on the calls that don’t need the big guns.
I still send the hard calls, the ones involving architecture or weird edge cases, to the strongest model I can buy. I’m not going to trust an 8B local run for a decision that costs me a weekend if it gets it wrong. But the easy stuff? Classification, summaries, simple transforms. That all sits on the cheap tier now, and the monthly bill dropped by about 70%.
The Problem
Sending everything to a premium cloud model is a fast way to light cash on fire. I had Kimi K2.5 handling research, first drafts, code review, and random one-off analysis. The bill ran $40-60 a month. Most of that work didn’t need a frontier model.
Old setup: everything through Kimi K2.5.
Result: ~$40-60/month in API calls.
New setup: route by task complexity.
Result: ~$12-15/month. Same output on the stuff that matters. Faster on the simple jobs.
The catch: you have to actually think about which task goes where. I keep a mental map and tweak it when something misroutes. That’s the cost of the 70%.
The Three-Tier System
I run three models and each one has a specific job. Quick stuff stays local and costs nothing. Mid-weight tasks hit a cheap cloud endpoint. The hard problems, the ones that need real reasoning or careful orchestration, go premium.
Yeah, it’s more complex than a single-model setup. Multiple endpoints to manage, more ways for things to break, and the local tier needs decent GPU power to feel responsive. It takes some babysitting. But the math works out: I’m spending roughly 70% less per month than I was when I sent everything through one premium model.
Here’s what I’m running right now. Prices come from vendor docs as of this post. Double-check current rates before you budget.
| Tier | Model | Cost | Use Case | Speed |
|---|---|---|---|---|
| —— | ——- | —— | ——―- | ——- |
| Fast/Free | Qwen3:14b (Ollama local) | $0 | Classification, summaries, simple drafting | Instant |
| Balanced | DeepSeek V3.2 (cloud) | ~$0.001/1K tokens | Content generation, research, code | 2-3s |
| Premium | Kimi K2.5 (cloud) | ~$0.003/1K tokens | Orchestration, complex reasoning, final polish | 5-8s |
What Each Tier Actually Does
All three run through Ollama on the same box. I route work based on how much thinking each task actually needs, not how shiny the output looks.
Fast Tier (Qwen3:14b)
- Spam email classification
- Topic tagging for articles
- Simple sentiment analysis
- Cron job status summaries
- Pre-filtering research results
Balanced Tier (DeepSeek V3.2)
- First drafts of blog posts
- Web research synthesis
- Code generation (Python scripts)
- Trading signal analysis
- Email response drafting
Premium Tier (Kimi K2.5)
- Final edit of published content
- Complex multi-step decisions
- Strategy recommendations
- Error analysis and debugging
- Orchestrator coordination
Three models means three things to keep updated, more VRAM than I’d like, and the occasional OOM kill when I forget to watch memory. Still worth it. The fast tier eats through most of the busywork without me thinking about it, and the premium one only gets called when I actually need the brainpower.
Real Routing Examples
Two pipelines I actually run. Same shape in both: cheap model filters, mid-tier does the real work, expensive one only fires when the decision matters.
TradeBot pipeline:
1. Scout checks 20 tokens → Qwen3 classifies “worth researching” vs “ignore” (free)
2. Researcher analyzes 5 tokens → DeepSeek evaluates momentum and risk ($0.002)
3. Orchestrator decides to buy 1 token → Kimi confirms the decision ($0.005)
Total: ~$0.007 per trade cycle vs. $0.025 if everything ran on Kimi
Content Nova pipeline:
1. Research 10 topics → Qwen3 picks the best 3 (free)
2. Write 3 article drafts → DeepSeek generates 1,200 words each ($0.04)
3. Final edit and SEO optimization → Kimi polishes ($0.02)
Total: ~$0.06 per article vs. $0.18 if everything ran on Kimi
The tradeoff: more moving parts. Three APIs to manage, each with their own rate limits, and when something breaks at 2am you’ve got three different error logs to dig through. I run monitoring on all of them now, but the first month of this setup was rough. I’d be lying if I said it wasn’t.
Honestly though, if you’re not pushing real volume, a single solid model is the easier path. I tier things because I’m processing thousands of calls a day. Most folks aren’t. Be honest about your scale before you take on the extra complexity.
The Hardware
Here’s what’s running the local tier. Just my desktop:
- AMD Ryzen 9800X3D (16 cores)
- Radeon 9070 XT (32 GB VRAM)
- 32 GB system RAM
- Windows 10
Qwen3:14b loads in about 4 seconds on this box. It chews through a 1K token prompt in roughly 2 seconds. Most days I forget I’m running local at all.
For the boring work — pulling data, reformatting output, running scripts — it feels identical to hitting a cloud API. The catch is VRAM. Push past a moderate context window or ask for a long output and that 32 GB ceiling shows up fast. You can quantize down and claw back some headroom, but quality takes a real hit. Not a trade I’d recommend for anything that matters.
When Local Models Fail
Hallucinations on factual tasks. Local models do this. Qwen3:14b will spit out fake URLs or pin quotes on people who never said them. DeepSeek and Kimi handle verification work better, from what I’ve gathered in community threads and vendor benchmarks. It’s the kind of thing that burns you once, then you stop trusting the model for anything citation-related.
Code quality drops on complex logic. Smaller models fumble on multi-file projects or API integrations. Kimi seems to produce cleaner output for that kind of work, though Qwen3 is fine if I’m writing a one-off Python script. The gap shows up around 200+ lines or when the logic branches hard.
Context window limits. Qwen3:14b sits at 32K, which covers most of my daily work. But a 50-page PDF or a long codebase blows past that fast. Kimi goes to 200K, which is why people reach for it on the heavy stuff. Honestly, the context window is the silent killer most folks don’t think about until they hit it.
The Cost Breakdown (Monthly)
| Category | Old (All Kimi) | New (Tiered) | Savings |
|---|---|---|---|
| ———- | —————- | ————– | ——— |
| Content generation | $25 | $8 | 68% |
| Trading research | $15 | $4 | 73% |
| Email/classification | $5 | $0 | 100% |
| Code/debugging | $10 | $3 | 70% |
| Total | $55 | $15 | 73% |
The GPU upfront cost stung. I’ll be the first to admit that. It’s a real line item before any of this even works. Once it’s paid off, the math flips in your favor fast.
That table above is my actual usage from last month. Kimi ran everything when I first set this up, and honestly, it worked fine. The catch: I was shelling out frontier prices for jobs a 7B local model handles just as well. Email triage, classification, quick code questions, basic rewrites. None of it needs the smartest model on the market.
So I tiered the workload. Heavy reasoning and anything that needs actual thinking stays on Kimi. Routine text production runs locally through Ollama on my own hardware. Middle-ground stuff hits cheaper API endpoints.
About $480 a year stays in my pocket. Covers the GPU and leaves room for a beer or two.
Implementation
Model routing isn’t some big project. It’s a config change. I run Ollama locally, and pointing each agent at a different model is a one-line edit:
yaml
agents:
tradebot-scout:
model: ollama/qwen3:14b
Fast, free
tradebot-researcher:
model: ollama/deepseek-v4-flash:cloud
Balanced
tradebot-orchestrator:
model: ollama/kimi-k2.6:cloud
Premium
Cron jobs decide which agent fires when. No code changes, no redeploys. That’s the upside. The catch: you still have to watch how each model actually performs at its task. The cheap local one isn’t great for nuanced reasoning, so I keep it scoped to simple lookups and route anything that needs real judgment up to the premium tier.
What’s Next
Three things on my list, in rough order of “how badly this will break”:
1. Dynamic routing: auto-detect task complexity, then send it to the cheapest model that can actually handle it
2. Fallback chains: if the local model fails, retry with cloud; if the mid-tier model falls short, escalate to premium
3. Quality scoring: log output quality per model per task type, then tweak routing over time
The catch with dynamic routing: it only works if your classifier is right. A bad guess means you pay for an extra model call on top of the wrong one. And nothing will bite you faster than a classifier that silently starts routing everything to your premium tier. That’s the part I’m not in a hurry to build.
The Lesson
I run three tiers. Ollama handles the cheap stuff on my own box, a mid-tier API does the work that needs to be roughly right, and the premium model only opens when the cheaper ones fail. Local eats about 70% of my workload.
Classification, summaries, rough drafts. All free-model territory. The premium tier earns its keep on maybe 10% of the work that actually justifies the bill.
There’s a tradeoff though. Routing logic adds real complexity, and if your volume is low, the savings won’t cover the setup time. This only pays off once you’re pushing meaningful volume through the pipeline.
Want the routing setup? Included in the Nova Operations blueprint ($49) – model configs, agent definitions, and cost tracking dashboard.
—
It’s not about pinching pennies. It’s about keeping costs from outpacing what the automation earns you.