Last updated 2026-05-17

Comparison vs alternatives

llmrouter is one of several reasonable answers to “how do I call an LLM from Go.” This page is an honest comparison against the alternatives, with explicit guidance on when to pick each one. We are biased — but the goal here is to make sure you do not adopt this library for a problem it was not built to solve.

The short version: if you only ever call one vendor and want every feature that vendor exposes, use that vendor’s official Go SDK. If you want a Python LiteLLM equivalent with typed wrappers around embeddings and tool use across ten providers, mozilla-ai/any-llm-go is closer to what you want. llmrouter exists for the specific case where you want a single OpenAI-shaped surface plus byte-level passthrough across OpenAI, Anthropic, and any OpenAI-compatible endpoint.

At a glance

The table below summarises the dimensions that usually decide the answer. “Byte passthrough” means: can you forward an upstream chunk to a downstream consumer without re-marshaling it. “Azure / Bedrock / Vertex” refers to the three managed cloud LLM endpoints that need different auth, body shape, and URL structure from the vendor’s own API.

vs. per-vendor SDKs (openai-go, anthropic-sdk-go, google.golang.org/genai)

Each major vendor publishes an official Go SDK. They are good libraries — stable, idiomatic, fully covering that vendor’s feature surface (tool calls, vision, audio, embeddings, assistants, batch APIs, etc.) the day each feature ships. If you only ever call one vendor, you should almost certainly use that vendor’s SDK.

Pros of per-vendor SDKs.

• Stable, post-1.0, officially supported.
• Complete coverage of that vendor’s API surface, including features that have not landed in any cross-vendor abstraction yet.
• Vendor-specific niceties — e.g. openai-go’s strongly typed tool-call helpers, anthropic-sdk-go’s helpers for the prompt-caching beta header.
• Vendor support channels actually know the SDK.

Cons.

• One SDK per provider. Your application code branches on which provider you’re calling — different types, different request shape, different stream events, different error wrappers.
• Hard to abstract away if you want to A/B test models across vendors or fail over from one to another.
• No native concept of “same API for OpenAI-compatible third parties.”
• No byte-level passthrough — these SDKs deserialize the wire format into their own types; if you want to proxy bytes you have to use the HTTP transport directly.

Pick the vendor SDK when:

• Your code only ever calls one provider, and you have no plan to add another.
• You need a feature that llmrouter does not model yet (embeddings, audio, vision, assistants, batch).
• You want the most stable possible API surface.

Pick llmrouter when:

• You call more than one vendor, and you don’t want to write provider-specific request building and stream parsing twice.
• You want to support “bring your own OpenAI-compatible endpoint” (OpenRouter, Together, Groq, vLLM, Ollama) without adding a new SDK per vendor.
• You want byte-identical pass-through for proxy/gateway use cases.

mozilla-ai/any-llm-go

mozilla-ai/any-llm-go is the closest neighbour to llmrouter in the Go ecosystem: a single API surface across multiple LLM providers, streaming-first, channel-based. It supports around ten providers out of the box including OpenAI, Anthropic, Gemini, Mistral, and several others.

If you want a Go equivalent of Python’s LiteLLM that wraps the most providers and gives you typed access to tool use and (in some cases) embeddings, mozilla-ai/any-llm-go ships more provider coverage today than llmrouter does.

Pros of any-llm-go.

• Broader provider catalogue out of the box.
• Typed wrappers around features beyond chat (where available).
• Backed by Mozilla AI, which gives it some institutional momentum.

Cons / gaps.

• No first-class Azure OpenAI Service, AWS Bedrock, or Google Vertex AI provider — the three managed cloud endpoints that most enterprise deployments end up using.
• No byte passthrough: requests and responses are deserialized into the library’s own typed shapes. You cannot run an any-llm-go-fronted proxy that is byte-identical to the upstream OpenAI wire format.
• Also pre-1.0; expect API changes.
• OpenAI is just one of many providers, not the canonical reference shape — so the request and response types are any-llm-go’s own design, not OpenAI’s.

Pick any-llm-go when:

• You need a provider llmrouter does not ship yet (e.g. Gemini direct, Mistral, Cohere) and waiting for v0.2 / v0.3 is not viable.
• You don’t care about byte-level passthrough and are happy consuming a library-specific typed surface.
• You want a closer Go analogue to Python’s LiteLLM in shape.

Pick llmrouter when:

• You specifically want the OpenAI request and response shape, not a third-library shape. Your existing OpenAI SDK code drops in.
• You are building a proxy, gateway, or telemetry layer that needs to forward bytes unchanged.
• You need Azure / Bedrock / Vertex on the roadmap with a single API surface, not three separate vendor SDKs.

LiteLLM and OpenLLM (Python, for reference)

LiteLLM and OpenLLM are the canonical Python answers to “one client, many LLMs.” They are excellent — LiteLLM in particular is the de facto standard if you are building in Python and want the OpenAI request shape against any provider, plus features like routing, budget caps, caching, and key rotation.

They are listed here only so you do not waste time evaluating them as Go options. If your codebase is Python, use LiteLLM. If it is Go, your choice is between llmrouter, per-vendor SDKs, any-llm-go, or direct HTTP.

Note that LiteLLM bundles a lot of gateway features (rate limiting, budget caps, caching, vector store integration) that llmrouter deliberately does not — see Out of scope on the roadmap page. Those belong in the gateway layer you build on top of a library like llmrouter, not in the library itself.

Direct HTTP (net/http + bufio.Scanner)

Every option above is a wrapper on top of net/http. You can skip them all and write your own client against each vendor’s REST endpoint. People do this — sometimes for good reasons.

Pros of direct HTTP.

• Zero dependencies beyond the standard library.
• Full control over retry, backoff, connection pooling, observability hooks, header injection, request mutation.
• No version churn imposed by a third-party library.
• If you only ever build the one client for the one path you care about, the total code can be smaller than any wrapper’s API surface.

Cons.

• You re-implement Server-Sent Events parsing per vendor. OpenAI uses standard SSE with data: lines and a sentinel [DONE]. Anthropic uses SSE with named event: types (message_start, content_block_delta, message_delta, message_stop). Each one is correct, none of them are the same.
• You re-implement the error envelope per vendor. OpenAI uses { "error": { "type": "...", "message": "..." } }. Anthropic uses { "type": "error", "error": { "type": "...", "message": "..." } }. Vertex returns Google-Cloud-shaped errors. Bedrock returns SigV4-style errors.
• You re-implement the request body per vendor — including, for Anthropic, the lifted system field and the content block shape.
• You write context-to-HTTP cancellation wiring, single-consumer channel discipline, and the producer-goroutine lifecycle every time.
• You re-do all of it for every new provider you add.

Pick direct HTTP when:

• You only ever call one endpoint, and the wrapper’s convenience does not pay for the dependency.
• You have a hard constraint against third-party dependencies.
• You want full control over the wire format because you’re doing something unusual (e.g. testing, fuzzing, debugging upstream behaviour).

Pick llmrouter when:

• You would otherwise write the same SSE parser, error wrapper, and producer goroutine twice — once per provider — and you’d like that code to be the same one library’s problem instead.

When llmrouter is the right choice

The library was built around four motivating use cases. If your project looks like one of these, llmrouter is probably the best fit:

1. OpenAI-shaped gateway for multiple upstreams. You expose an OpenAI-shaped HTTP API to your callers, but internally you route to OpenAI, Anthropic, OpenRouter, or a self-hosted vLLM. With llmrouter your handler does not branch on provider: it builds a single ChatRequest, picks a provider, and streams the result back. Forwarding Chunk.Raw downstream keeps the response byte-identical to upstream OpenAI for callers that have strict expectations of the SSE format.
2. Bring your own OpenAI-compatible endpoint. Your users want to point your product at OpenRouter, Together, Groq, vLLM, Ollama, LM Studio, or a private model behind their firewall. With llmrouter this is one WithBaseURL away — no new SDK per endpoint.
3. Multi-vendor application code. Your application calls both OpenAI and Anthropic from the same code path (model A/B tests, failover, “cheap fast model then expensive smart model” cascades). You don’t want provider-specific branches inside your business logic; you want the provider abstracted behind one interface.
4. Production streaming with proper cancellation. You run a long-lived server that opens streams to upstream LLMs. You need context.Context cancellation that reaches the in-flight HTTP request (so a client hang-up actually kills the upstream connection, freeing budget). The stream semantics in llmrouter are designed for this.

When llmrouter is the wrong choice (honest weaknesses)

The library is pre-1.0 and intentionally narrow. We’d rather you know about the gaps than discover them after a migration.

• Pre-1.0 API churn. We expect minor breakages between minor versions (v0.1 → v0.2 → v0.3) before we freeze the API at v1.0. If you need a library you can bump without reading changelogs, wait until v1.0 — or pin a minor and absorb a migration each time you upgrade.
• Missing providers. v0.1 ships OpenAI and Anthropic. Azure OpenAI Service, AWS Bedrock, and Google Vertex AI are planned for v0.2 but not shipped yet. Direct Gemini, Mistral, Cohere are not on the near-term roadmap — for those, mozilla-ai/any-llm-go ships today.
• No embeddings API. The library is focused on streaming chat completions. Embed() is on the v0.3 roadmap. If you need embeddings now, use the vendor SDK.
• No audio / TTS / STT API. Whisper, GPT-4o audio, and Vertex audio are v0.4 material. Today you would call them with the vendor SDK directly.
• No vision provider abstraction yet. You can send vision content today via ChatRequest.Raw byte passthrough, but there is no typed helper for image_url / image bytes content blocks. That lands in v0.3.
• No tool-call typed surface yet. Same as vision: you can pass tool definitions and consume tool calls via byte passthrough, but the typed ChatRequest.Tools / Choice.Message.ToolCalls shape lands in v0.3.
• No gateway features. Rate limiting, budget caps, response caching, key rotation, vector store integration, persistent conversation state — these are all intentionally out of scope. They belong in a gateway layer you build on top of the library. If you want them bundled, LiteLLM (Python) is closer to that shape.

Decision tree

A short, honest version of the above:

• Only ever calling one vendor, want the full vendor surface? → that vendor’s official Go SDK.
• Need embeddings, audio, or assistants APIs today? → the vendor SDK (or any-llm-go for embeddings coverage).
• Need Gemini / Mistral / Cohere / a long tail of providers now? → any-llm-go.
• Calling OpenAI + Anthropic + OpenAI-compatible endpoints from one codebase, want OpenAI shape, want byte passthrough, can live with v0.x churn? → llmrouter.
• Building a proxy / gateway that forwards SSE byte-identical to upstream? → llmrouter (this is the headline use case).
• Allergic to dependencies and only need one provider? → direct net/http.

If we got something wrong

Comparisons go stale. If something on this page is out of date, or you think we’ve mis-characterised an alternative, please open an issue at github.com/elloloop/llmrouter/issues with the corrected facts. We’d rather have an accurate comparison page than a flattering one.