OpenAI SDK kit

The catalog's openai-compatible base URLs work with the stock OpenAI SDKs — that's the selection criterion. Two patterns:

Pattern A — gateway (cascade, cooldowns, breakers handled for you)

import OpenAI from "openai";
const client = new OpenAI({
  baseURL: "http://localhost:8787/v1",
  apiKey: "local",
});
const res = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "hello" }],
});
console.log(res.choices[0].message.content);

from openai import OpenAI
client = OpenAI(base_url="http://localhost:8787/v1", api_key="local")
res = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "hello"}],
)

Pattern B — direct provider

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.groq.com/openai/v1",   # from: pnpm mh show groq
    api_key=os.environ["GROQ_API_KEY"],
)
res = client.chat.completions.create(
    model="llama-3.3-70b-versatile",
    messages=[{"role": "user", "content": "hello"}],
    max_tokens=256,
)

Base URLs that work this way today (from the catalog): Groq, Google AI Studio (.../v1beta/openai), OpenRouter, NVIDIA NIM (https://integrate.api.nvidia.com/v1), Cerebras, Mistral, Cohere (/compatibility/v1), HuggingFace router, OVHcloud, Z AI, GitHub Models, and the trial-credit tier. pnpm mh list --openai-compatible --json is the machine-readable list.

Gotchas

• "OpenAI-compatible" ≠ identical semantics: tool calling, response_format, and vision payloads vary per provider. Probe before you trust: pnpm mh probe --live --provider X.
• Thinking models (e.g. gemini-2.5-flash) consume max_tokens on reasoning — a tiny cap can return finish_reason: "length" with empty content. Use ≥64 for smoke tests.
• 429 handling differs (some send Retry-After, some don't). The gateway normalizes this; raw SDK usage should back off exponentially.