FreeLLMAPI: Self-Hosted Router for 34 Free LLM Providers

Q: How does FreeLLMAPI handle rate limits across different providers?

The router tracks RPM, RPD, TPM, and TPD counters per (platform, model, key) triple. It learns each provider's reported ceilings from live traffic, cools down keys that hit a cap, and retries the next viable model in your fallback chain. Background health checks classify keys as healthy, rate-limited, invalid, or error, skipping non-healthy keys without making a live request.

Q: What happens when a provider changes its free-tier quota or drops a model?

The signed catalog from freellmapi.co syncs twice daily. Quota changes, new models, and provider quirk fixes are applied automatically. Free installs receive these updates via the monthly snapshot (30-day delay); Premium routers get them same-day. Your custom providers and enable/disable choices are never overwritten. FreeLLMAPI doesn't create more free tokens, it makes the ones you already have across 34 providers usable without the manual overhead. For personal experimentation, rapid prototyping, and agent loops where cost matters more than predictable latency, that trade-off is often the right one.

How FreeLLMAPI Turns Fragmented Free Tiers Into a Single OpenAI-Compatible Endpoint

FreeLLMAPI solves the operational headache of juggling dozens of free LLM accounts by running a local router that pools 34 providers behind one /v1 endpoint. The project encrypts your provider keys at rest, tracks per-key rate limits in real time, and fails over automatically when a provider hits its quota. A signed model catalog from freellmapi.co keeps the router current, free installs receive a monthly snapshot, while a $19/year Premium tier delivers same-day updates. The result is roughly 7.4 billion tokens per month of aggregate free capacity across 635 model endpoints from 474 model families, all accessible through any OpenAI-compatible client.

The Free-Tier Fragmentation Problem

Every major AI lab now offers a free tier, Google AI Studio, Groq, Cerebras, Mistral, NVIDIA NIM, OpenRouter, Cohere, Cloudflare Workers AI, Z.ai, Hugging Face, and more. On paper the numbers look impressive: OpenRouter alone routes across 70+ providers and processes 100 trillion tokens per month OpenRouter Blog. In practice, each provider demands its own API key, dashboard, rate-limit schema, and model naming convention. A coding agent that runs for ten minutes can stall when one provider hits a daily cap, forcing you to swap keys, change model names, and restart the task.

The rate-limit landscape is opaque. As of 26 July 2026, only Groq publishes a complete per-model free-tier table covering RPM, RPD, TPM, and TPD Shahriar Labs. Google AI Studio and Gemini direct users to a console view that varies by account; Mistral refers to an admin console; Cerebras publishes no free-tier figures at all and notes its paid Developer tier offers 10x higher rate limits; NVIDIA NIM meters access in build credits rather than standard rate metrics. This inconsistency makes manual orchestration brittle.

How FreeLLMAPI Works

Architecture Overview

The router is a TypeScript application that runs on Node 20+Windows, macOS, Linux, or ARM single-board computers like a Raspberry Pi. At idle it consumes roughly 40 MB RSS. Provider keys are stored in a local SQLite database encrypted with AES-256-GCM and decrypted only in memory per request. Your applications never see the upstream keys; they authenticate with a single unified freellmapi-… bearer token generated by the dashboard.

Routing and Failover

Six routing strategies ship out of the box: Manual, Balanced, Smartest, Fastest, Most Reliable, and Custom. The Balanced strategy (recommended default) weights reliability at 50%, speed at 25%, and intelligence at 25% Better Stack Community. The router maintains live per-model scores for each dimension, tracks RPM/RPD/TPM/TPD counters per (platform, model, key) triple, and cools down keys that hit a cap. On a 429 or 5xx response it retries the next viable model in your fallback chain.

Model Catalog Synchronization

The router pulls a signed catalog from freellmapi.co twice daily. The catalog currently tracks 34 providers, 474 model families, and 635 free endpoints (584 chat, 41 embeddings, 7 transcription, 3 video). Free installs receive the monthly snapshot, a model joins the snapshot 30 days after it lands in the live feed. Premium routers ($19/year or $49 lifetime) get the live feed the same day changes ship. Your enable/disable choices and custom providers are never overwritten.

FreeLLMAPI: Self-Hosted Router Aggregates 34 Free LLM Providers

Metric	Free Install	Premium
Catalog freshness	Monthly snapshot (~30-day lag)	Live feed (same day)
Models in catalog	~332 (303 behind live)	635
Providers tracked	34	34
Aggregate free tokens/month	7.4B	7.4B
Price	$0	$19/yr or $49 lifetime

API Surface Compatibility

FreeLLMAPI exposes the full OpenAI-compatible surface: /v1/chat/completions, /v1/responses (for Codex CLI), /v1/completions (editor ghost-text), /v1/images/generations, /v1/videos/generations, /v1/audio/speech, /v1/audio/transcriptions, /v1/embeddings, and /v1/modelsstreaming and non-streaming. It also implements the Anthropic Messages API at /v1/messages so Claude Code and the official Anthropic SDKs work unchanged. Native Gemini wire format is available at /v1beta (generateContent, streaming, token counting, models), and an opt-in Ollama emulation serves NDJSON chat/generate, tags, metadata, and embeddings for Zed, JetBrains AI, and other local-model clients.

635 free model endpoints across 34 providers aggregate to roughly 7.4 billion tokens per month of listed free-tier capacity.FreeLLMAPI GitHub README

Real Benefits for Development Workflows

Eliminating Key Management Friction

Instead of maintaining separate environment variables for Groq, Google AI Studio, Cerebras, Mistral, and two dozen others, you add each key once in the FreeLLMAPI dashboard. The router verifies each key, shows a health status dot, and surfaces last-checked timestamps. A unified key from the Keys page header is the only credential your scripts and tools need.

Automatic Failover Keeps Agents Running

When a provider hits a rate limit, the router cools that key down and retries the next model in your chain without interrupting the calling code. This is critical for agentic workflows where a task runs for minutes, a mid-task quota exhaustion no longer stalls the entire pipeline. Sticky sessions keep a conversation on the same model for 30 minutes; an optional compact handoff note preserves coherence when a mid-chat switch does occur.

Multi-Model Synthesis With Fusion

The virtual fusion model fans your prompt out to a panel of diverse free models in parallel, then a judge model synthesizes one answer from the drafts. This can improve reasoning quality over any single free model, albeit with higher latency and token consumption across the panel.

Prompt Compression and Response Caching (Opt-In)

A shared, fail-open request pipeline can deduplicate prompts, filter tool output, compact repeated JSON, and trim stale context before cache lookup and routing. Response caching is also opt-in. Both features reduce upstream token usage and latency without changing your application code.

Practical Examples

Docker One-Liner Install

curl -fsSL https://freellmapi.co/install.sh | bash

The script sets up ~/freellmapi, generates an encryption key, pulls the ghcr.io/tashfeenahmed/freellmapi:latest image, and starts the container. Re-running is safe, your .env and encryption key are preserved. The dashboard opens at http://localhost:3001.

Desktop App for Zero-Config Local Use

Native menu-bar apps for macOS (.dmg) and Windows (.exe installer) ship with every release on GitHub Releases. No account or password setup, the dashboard signs itself in with a hidden local account. The tray popover shows live request stats and your unified API key.

Configuring Coding Agents

Automated setup generators fetch your live catalog, back up existing config, and merge changes without clobbering:

npx freellmapi setup-claude --url http://localhost:3001 --api-key <unified-key>
npx freellmapi setup-codex --url http://localhost:3001 --api-key <unified-key>
npx freellmapi setup-aider, setup-cline, setup-continue, setup-opencode, setup-goose, setup-qwen, setup-roo, setup-kilo, setup-crush, setup-dsh (DeepSeek Harness), setup-mimo, setup-cursor

Launchers keep credentials out of config files entirely: npx freellmapi launch for Claude Code and npx freellmapi launch-codex for Codex inject credentials into the child process only.

Python Usage Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:3001/v1",
    api_key="freellmapi-your-unified-key",
)

resp = client.chat.completions.create(
    model="auto",  # or "auto:fast", "auto:smart", a profile, or a model id
    messages=[{"role": "user", "content": "Summarise the fall of Rome in one sentence."}],
)
print(resp.choices[0].message.content)
print("Routed via:", resp.headers.get("x-routed-via"))

Every response carries an X-Routed-Via: <platform>/<model> header so you can audit which provider actually served the request.

Limitations and Trade-Offs

Stacking free tiers has real constraints. No frontier models are available, the catalog tops out at strong open-weight models like Qwen3 Coder, DeepSeek V3/R1, Kimi K2, GLM, Nemotron, and GPT-OSS. Latency varies significantly between providers; a request routed to Groq may return in milliseconds while the next falls back to a slower endpoint. There is no SLA, and effective intelligence dips late in the day as top models hit daily caps, resetting at UTC midnight. The project explicitly states it is for personal experimentation and learning, not production. If you build something real on FreeLLMAPI, swap in a paid API before you ship.

Each upstream provider’s terms of service still apply when traffic is proxied through FreeLLMAPI. A provider-by-provider ToS review (conducted May 2026) lives in docs/architecture.md#terms-of-service-review. Notably, Cohere’s free tier bans “personal, family, or household purposes” WotAI, and roughly half of the ~110 free models across 16 providers surveyed in June 2026 have terms that restrict personal use WotAI.

FreeLLMAPI

Open-source, self-hosted LLM router aggregating 34 free-tier providers behind one OpenAI-compatible /v1 endpoint.

Unified API key

A single freellmapi-… bearer token your apps use; upstream provider keys stay encrypted locally.

Fusion

Virtual model that fans a prompt to multiple free models in parallel and synthesizes a single answer via a judge model.

Live catalog vs. monthly snapshot

Premium routers receive catalog updates (new models, quota changes, fixes) same-day; free installs get a 30-day-delayed snapshot.

Sticky sessions

Conversations stay on one model for 30 minutes; optional compact handoff note preserves context on forced switches.

How It Compares to Alternatives

OpenRouter is a hosted, managed service, your requests pass through their platform. FreeLLMAPI runs locally; your provider keys never leave your machine. LiteLLM is production-oriented with caching, virtual keys, spending limits, and a dashboard on port 4000 Tencent Cloud. FreeLLMAPI has a narrower focus: free-tier maximization for personal development. Bifrost targets regulated industries with native MCP support, VPC isolation, and 11 µs overhead at 5,000 RPS GetMaxim. FreeLLMAPI prioritizes simplicity and privacy over enterprise governance.

📊 What This Means for Developers in 2026

Free-tier aggregation is now practical: FreeLLMAPI turns 34 fragmented free tiers into one endpoint you can point any OpenAI client at, with automatic failover and per-key quota tracking.
Catalog freshness is the differentiator: The 30-day lag on free installs matters when providers launch or retire models weekly. Premium at $19/year is a low-cost hedge if you rely on the latest free models.
Local-first security wins for experimentation: Provider keys encrypted at rest with AES-256-GCM, decrypted only in memory per request, and a single unified token for your tools, no third party sees your credentials or prompts.
Don’t confuse prototyping capacity with production reliability: Variable latency, no SLA, daily cap resets at UTC midnight, and ToS restrictions on several providers mean this is a development accelerator, not an inference substrate for user-facing apps.
The routing engine is the real product: Six strategies, live reliability/speed/intelligence scores, sticky sessions, Fusion synthesis, and prompt compression make the router more than a simple proxy, it’s a free-tier optimizer.

FreeLLMAPI: Self-Hosted Router Aggregates 34 Free LLM Providers