The free chat cascade
The default answer to "I want free chat completions that keep working when one provider throttles me." Ordered by tier durability and quota depth from the ModelHubby catalog (2026-07 snapshot), not by hype.
Personalize this:
pnpm mh recommend --snippetranks against your env keys and live health data. The chain below assumes you've collected keys from the top free tiers (each takes ~5 minutes — seecontent/guides/).
The chain
| # | Provider | Model | Why this rung |
|---|---|---|---|
| 1 | Groq | llama-3.3-70b-versatile |
Deep free tier (30 RPM / 14,400 RPD), very fast, no card |
| 2 | Google AI Studio | gemini-2.5-flash |
Strong model, generous free tier — but free-tier prompts may be used for training outside UK/CH/EEA/EU |
| 3 | Cerebras | gpt-oss-120b |
Extremely fast; official free-tier figures are tighter than folklore (5 RPM per official page) |
| 4 | OpenRouter | meta-llama/llama-3.3-70b-instruct:free |
Aggregator safety net: 50 RPD free (1,000 RPD after one-time $10 top-up); free upstreams may log prompts |
| 5 | OVHcloud | gpt-oss-120b |
Anonymous, zero-signup fallback — ~2 RPM/IP, but it answers when everything else is exhausted |
Wire it
Run the gateway and let it do the fallback, breakers, and key cooldowns for you:
export GROQ_API_KEY=... GEMINI_API_KEY=... CEREBRAS_API_KEY=... OPENROUTER_API_KEY=...
pnpm mh serve
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "http://localhost:8787/v1",
apiKey: "unused-local",
});
const res = await client.chat.completions.create({
model: "auto", // ModelHubby routes; check x-modelhubby-provider on the response
messages: [{ role: "user", content: "hello" }],
});
No gateway? Export the standalone chain: pnpm mh recommend --snippet emits a
self-contained completeWithFallback() with your ranked rungs baked in.
Failure semantics you get for free
- 429 → key cools down (30s doubling, 6h cap), next rung fires immediately
- 402 → key parked until quota reset; rung skipped
- 401/403 → key quarantined (
auth_suspect), never retried blindly - 5 failures/60s on one provider → circuit opens 120s
bad_request→ fails fast to you (retrying other rungs can't fix your payload)