Gateway drop-in

Anything that speaks the OpenAI chat API can route through ModelHubby and inherit the cascade, key cooldowns, and circuit breakers. One base URL swap.

pnpm mh serve          # http://localhost:8787

OpenAI SDK (any language)

const client = new OpenAI({
  baseURL: "http://localhost:8787/v1",
  apiKey: "local",
});
// model: "auto"                        → router picks (policy: free-only by default)
// model: "groq/llama-3.3-70b-versatile" → pin provider/model
// model: "llama-3.3-70b-versatile"      → pin model, any provider that serves it

Policy per request (headers)

curl http://localhost:8787/v1/chat/completions \
  -H 'content-type: application/json' \
  -H 'x-mh-no-training: true' \
  -H 'x-mh-free-only: true' \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "hi"}]}'

x-mh-free-only · x-mh-no-card · x-mh-no-training — same semantics as the CLI flags; "unknown" providers never satisfy a safety header.

Coding agents (Cursor, Cline, Continue, …)

Any tool with "OpenAI-compatible / custom base URL" settings:

• Base URL: http://localhost:8787/v1
• API key: anything non-empty (the gateway holds the real keys via your env)
• Model: auto, or pin one

The response headers tell you what actually served each request: x-modelhubby-provider, x-modelhubby-model.

What the gateway will not do

• Proxy for other people (local-first; no auth layer — do not expose the port)
• Translate non-OpenAI payload dialects (Anthropic-native clients need an anthropic-compatible rung — not in v1)
• Persist your prompts (nothing is logged beyond health/key state; fingerprints only)