OpenRouter vs Groq vs Together AI: What Inference Costs
Somewhere between your prototype and your first paying customer, the model bill stops being funny. I hit that wall in a boring way: a support bot that cost nothing to demo started costing real money at a few thousand messages a day, and I had no idea which of the three obvious vendors was cheapest for my shape of traffic. So I sat down with the pricing pages.
Disclosure: This article contains affiliate links. If you buy through one of them, this site may earn a commission at no extra cost to you.
What follows is not a benchmark. It is a billing-model comparison, because that is where the money actually goes. OpenRouter, Groq, and Together AI all sell you tokens from open-weight models, and they all land in the same rough price band. The differences are in how they charge you, what happens when a model disappears, and how much of your monthly bill is fees rather than inference.
The short answer
- OpenRouter is the aggregator. You get 600+ models behind one key, and inference is billed at the provider’s own list price with no markup. The cost is a 5.5% fee every time you buy credits (8% on the Business plan).
- Groq is the speed play. Same open-weight models, dramatically lower latency, but a much shorter model list than it had a year ago, and a hard free-tier ceiling of 8,000 tokens a minute.
- Together AI is the catalogue play. Widest set of open models plus fine-tuning from roughly $0.34 per million training tokens, with no platform fee at all.
How each one actually bills you
OpenRouter: no markup, but a fee at the till
This is the part people get wrong. OpenRouter does not mark up tokens. If a model costs $0.15 per million input tokens from the lab, you pay $0.15. The revenue comes from the credit purchase itself: 5.5% on the Standard pay-as-you-go plan, 8% on Business, and a negotiated rate on Enterprise.
That fee is small in absolute terms and large in relative ones. On $50 a month of inference you are paying $2.75 in fees, which nobody cares about. On $2,000 a month you are paying $110, and at that point writing a second integration against a direct provider starts to look like paid work.
There is a genuinely good wrinkle for anyone already holding provider keys. Bring-your-own-key on OpenRouter is free up to $25,000 of list-price inference per month, then 5% after that. If you already have an Anthropic or OpenAI account, you can route through OpenRouter without paying the credit fee on that traffic.
Groq: per-token, with the cheapest fast model on the board
Groq runs open-weight models on its own LPU hardware and charges straight per token, no platform fee. The current self-serve lineup is short, and Groq’s own docs list prices that the marketing site no longer shows. As of late October 2026 the numbers were:
- GPT-OSS 120B at $0.15 input, $0.075 cached input, $0.60 output, around 500 tokens per second
- GPT-OSS 20B at $0.075 input, $0.037 cached, $0.30 output, around 1,000 tokens per second
- Qwen 3.8 27B at $0.80 input and $4.00 output, the only self-serve model that takes images
Batch jobs get 50% off every token. Cached input is half price. Those two discounts stack against a workload nicely, and they are the reason Groq’s headline rate can look unremarkable while the effective rate is not.
The free tier is real but tight: 30 requests and 8,000 tokens per minute on GPT-OSS 120B. That is enough to build against and not enough to launch on.
Together AI: the widest catalogue, no platform fee
Together bills per token like Groq, and its serverless list is the longest of the three. Serverless prices checked in October 2026 ran from $0.09 per million input on Qwen3.8 Flash up to $3.00 input and $15.00 output on Kimi K3. GPT-OSS 120B sits at the same $0.15 and $0.60 as Groq. DeepSeek V4.1 Flash was $0.30 input and $1.20 output.
The differentiator is fine-tuning. Together prices training from about $0.34 to $100 per million training tokens depending on model size and method, and fine-tuned inference runs at roughly three times the base rate. If your product depends on a model that has learned your support macros, that is the cheapest path I found.
Side by side
| Provider | Billing model | Platform fee | GPT-OSS 120B (in / out per 1M) | Free tier |
|---|---|---|---|---|
| OpenRouter | Credits, provider list price | 5.5% on credit purchase (8% Business) | $0.15 / $0.60 | 25+ free models, 50 requests/day |
| Groq | Straight per token | None | $0.15 / $0.60 | 30 req/min, 8,000 tokens/min |
| Together AI | Straight per token | None | $0.15 / $0.60 | Signup credits, amount varies |
Read that table again and notice the interesting bit. The same model is the same price at all three. What you are choosing between is a fee, a speed profile, and a model list.
What a real workload costs
Take a support bot handling 20 million input tokens and 4 million output tokens a month. Not huge. Roughly the volume of a small SaaS with a chat widget on every page.
- Groq, uncached: 20 x $0.15 plus 4 x $0.60 comes to $5.40 a month.
- Groq, with 60% of input cached: 12 x $0.075 plus 8 x $0.15 plus $2.40 comes to $4.50.
- Together, same model: $5.40, no fee on top.
- OpenRouter, same model: $5.70 once the 5.5% credit fee is included.
Thirty cents a month is not a decision. At 40 times that volume, the gap is about $12 a month, still not a decision. The fee only matters when your inference bill is large enough that 5.5% is a line item you would notice losing, which in practice means somewhere north of a thousand dollars a month.
Where the real money hides is in model choice, not routing. Running the same bot on DeepSeek V4.1 Flash instead of GPT-OSS 120B doubles the input cost and doubles the output cost. Running it on Kimi K3 multiplies both by twenty. No amount of fee optimization survives a bad model pick, which is why I check the model price before I check the router.
Where Groq actually earns its place
Latency. If a human is waiting on the other end of a chat box, 500 tokens per second changes the feel of the product in a way that a cheaper model never will. I would not run batch enrichment on Groq, but the interactive surface is a different question.
The tradeoff is volatility. Groq has removed models from its self-serve plans repeatedly through 2026, including Llama 3.1 8B, Llama 3.3 70B, and MiniMax M2.7, which now sit behind contact-sales. A model you build on in March can be a procurement conversation by August. That is a real architectural risk and it does not show up on any pricing page.
Where Together earns its place
Breadth, and the ability to stop renting someone else’s personality. If you are going to fine-tune, Together is the only one of the three where the training cost is small enough to treat as an experiment rather than a project. The catch is that fine-tuned inference runs at roughly triple the base rate, so the math only works if the smaller tuned model replaces a bigger general one.
Where OpenRouter earns its place
Fallback. One key, one bill, and if a provider degrades you can reroute rather than page yourself at 2am. That is worth more than 5.5% to a one-person company, because the alternative is maintaining three SDK integrations you will not keep current.
The thing that surprised me when I mapped this out: OpenRouter’s model coverage is not uniform in quality. Plenty of models on the aggregator are served by exactly one provider, so the “route around a bad provider” promise quietly does not apply to them. The routing is a genuine feature, not a universal one.
How I would choose, honestly
I run most of my own inference locally on Ollama, which is free and slow and the right answer for background jobs that can wait. When I need cloud capacity, the decision tree is short:
- Prototyping and comparing models: OpenRouter. The fee is noise at this stage and the model switching is the whole point.
- Anything a user waits on: Groq, if your chosen model is still self-serve there. Verify that first, because the list shrinks.
- Fine-tuning, or a long-tail open model: Together.
- Over $1,000 a month of steady inference: go direct to the lab you actually use. The 5.5% is now a salary line.
One caveat on all of this: none of these numbers are load-bearing for more than a quarter. Groq’s own pricing page stopped publishing rates and moved them into developer docs. Together’s serverless list has dropped models I would have called permanent in April. Check the vendor page the week you commit, not the week you read a comparison.
If you want the deeper cost math on running models yourself versus paying per token, I wrote up the local versus cloud cost comparison, and a separate piece on picking a local runtime. Both are the cheaper end of this decision.
Key takeaways
- All three charge the same per-token price for the same model. The difference is the fee, the speed, and the catalogue.
- OpenRouter’s 5.5% credit fee is irrelevant under a few hundred dollars a month and material above a thousand.
- Groq’s cached input at half price and 50% batch discount can beat a nominally cheaper provider on the same workload.
- Groq’s model list is volatile. Do not build a dependency on a preview model there.
- Together is the only one of the three where fine-tuning is cheap enough to try on a whim.
- Model choice moves your bill by 2x to 20x. Router choice moves it by 5%.
Try them in this order
Start with OpenRouter’s free tier, because 25 free models and 50 requests a day will tell you which model you actually want without a card on file. Then take that model to Groq if latency matters, and to Together if you need something the other two stopped serving. Run the same 100 prompts through each and compare the replies before you compare the invoices. The cheapest model that fails your quality bar is the most expensive thing you can buy.
