Last updated 2026-05-17

Package router

Every public symbol declared under router/. This package resolves {Model, Platform, Credentials} triples into concrete llmrouter.Provider implementations. See the Router concept for the design narrative and worked examples.

Import path:

import "github.com/elloloop/llmrouter/router"

Resolve

Source: router/router.go#L54.

func Resolve(req Request) (llmrouter.Provider, error)

Builds an llmrouter.Provider for the requested {Model, Platform, Credentials} tuple. Steps:

  1. Validates req.Model is non-empty.
  2. Calls InferFamily(req.Model) to pick a ModelFamily.
  3. If req.Platform == PlatformAuto, scans autoPreference and picks the first platform that supports the family and has credentials.
  4. Validates the (family, platform) pair against SupportedPlatforms(family).
  5. Dispatches to the right underlying provider constructor and returns the result.

Returns one of: ErrEmptyModel, ErrUnsupportedRoute, ErrMissingCredentials, ErrNoAutoPlatform, or a wrapped llmrouter.ErrInvalidConfig from the underlying provider's New().

ResolveFromEnv

Source: router/router.go#L75.

func ResolveFromEnv(model string) (llmrouter.Provider, error)

Convenience for the common case: equivalent to Resolve(Request{Model: model, Platform: PlatformAuto}). Reads all credentials from environment variables according to DefaultEnvVars.

InferFamily

Source: router/family.go#L32.

func InferFamily(model string) ModelFamily

Returns the ModelFamily implied by a model id via case-insensitive prefix matching after stripping hosting-platform vendor prefixes (anthropic., meta., mistral., cohere., amazon., ai21., stability.). Unknown ids return FamilyOther.

SupportedPlatforms

Source: router/platform.go#L62.

func SupportedPlatforms(family ModelFamily) []Platform

Returns the platforms that can serve the given family, in the router's preferred order. Useful for building custom preference loops when PlatformAuto isn't quite right.

ApplyModelTranslation

Source: router/router.go#L330.

func ApplyModelTranslation(req llmrouter.ChatRequest, platform Platform) llmrouter.ChatRequest

Returns a copy of req with Model rewritten to the platform-specific form. Currently translates Anthropic and Llama / Mistral / Cohere ids to the Bedrock prefixed form (anthropic.claude-3-5-sonnet-20241022-v2:0) and Anthropic ids to the Vertex @-versioned form (claude-3-5-sonnet-v2@20241022). The original request is unchanged. Ids that already carry the right prefix or @-version pass through unchanged.

Platform

Source: router/platform.go.

type Platform string
const (
PlatformAuto Platform = "" // pick from autoPreference based on credentials
PlatformDirect Platform = "direct" // vendor's native API
PlatformBedrock Platform = "bedrock" // AWS Bedrock Runtime ConverseStream
PlatformVertex Platform = "vertex" // Google Vertex AI / Model Garden
PlatformAzure Platform = "azure" // Azure AI Foundry (router picks sub-provider)
PlatformOpenRouter Platform = "openrouter" // OpenRouter fan-out proxy
PlatformTogether Platform = "together" // Together AI
PlatformGroq Platform = "groq" // Groq Cloud
PlatformFireworks Platform = "fireworks" // Fireworks AI
PlatformCerebras Platform = "cerebras" // Cerebras Cloud Inference
PlatformDeepSeek Platform = "deepseek" // DeepSeek first-party API
PlatformPerplexity Platform = "perplexity" // Perplexity sonar-family API
PlatformxAI Platform = "xai" // xAI Grok API
)

ModelFamily

Source: router/family.go.

type ModelFamily string
const (
FamilyOpenAI ModelFamily = "openai" // gpt-*, o1-*, o3-*, o4-*, chatgpt-*
FamilyAnthropic ModelFamily = "anthropic" // claude-*
FamilyLlama ModelFamily = "llama" // llama-*, meta.llama-*, anything containing "llama"
FamilyMistral ModelFamily = "mistral" // mistral-*, mixtral-*, ministral-*, codestral-*, magistral-*, pixtral-*
FamilyCohere ModelFamily = "cohere" // command-*, c4ai-*
FamilyGemini ModelFamily = "gemini" // gemini-*
FamilyGrok ModelFamily = "grok" // grok-*
FamilyDeepSeek ModelFamily = "deepseek" // deepseek-*
FamilyOther ModelFamily = "other" // unknown / opaque
)

Credentials

Source: router/credentials.go#L12.

type Credentials struct {
// Generic API key for direct platforms and most OpenAI-compatible
// proxies. Empty falls back to a family- or platform-specific env var.
APIKey string
// Azure-specific fields.
AzureBaseURL string // resource hostname; e.g. https://<resource>.services.ai.azure.com
AzureDeployment string // deployment name from the Foundry portal
AzureAPIVersion string // e.g. "2024-10-21"
AzureAPIKey string // overrides APIKey for Azure-only setups
// AWS Bedrock. The AWS SDK handles access-key wiring via its
// standard credential chain — the router only needs the region.
AWSRegion string
// GCP. Project + region select the Vertex tenant. Access token is
// optional; when empty, Application Default Credentials are used
// by the underlying provider.
GCPProject string
GCPRegion string
GCPAccessToken string
}

Request

Source: router/credentials.go#L103.

type Request struct {
Model string // required; e.g. "claude-3-5-sonnet-20241022"
Platform Platform // PlatformAuto picks from autoPreference
Credentials Credentials // platform-specific auth fields
HTTPClient *http.Client // optional; forwarded when the provider supports it
}

EnvVars

Source: router/credentials.go#L48.

type EnvVars struct {
// Family-keyed API keys for direct platforms.
OpenAIAPIKey string
AnthropicAPIKey string
MistralAPIKey string
CohereAPIKey string
GoogleAPIKey string // Gemini direct
GrokAPIKey string // xAI direct
DeepSeekAPIKey string
// Platform-keyed API keys for fan-out platforms.
OpenRouterAPIKey string
TogetherAPIKey string
GroqAPIKey string
FireworksAPIKey string
CerebrasAPIKey string
PerplexityAPIKey string
// Azure / AWS / GCP.
AzureBaseURL string
AzureAPIKey string
AWSRegion string // primary
AWSRegionAlt string // fallback (typically AWS_DEFAULT_REGION)
GCPProject string
}

DefaultEnvVars

Source: router/credentials.go#L77.

var DefaultEnvVars = EnvVars{
OpenAIAPIKey: "OPENAI_API_KEY",
AnthropicAPIKey: "ANTHROPIC_API_KEY",
MistralAPIKey: "MISTRAL_API_KEY",
CohereAPIKey: "COHERE_API_KEY",
GoogleAPIKey: "GOOGLE_API_KEY",
GrokAPIKey: "GROK_API_KEY",
DeepSeekAPIKey: "DEEPSEEK_API_KEY",
OpenRouterAPIKey: "OPENROUTER_API_KEY",
TogetherAPIKey: "TOGETHER_API_KEY",
GroqAPIKey: "GROQ_API_KEY",
FireworksAPIKey: "FIREWORKS_API_KEY",
CerebrasAPIKey: "CEREBRAS_API_KEY",
PerplexityAPIKey: "PERPLEXITY_API_KEY",
AzureBaseURL: "AZURE_OPENAI_ENDPOINT",
AzureAPIKey: "AZURE_OPENAI_API_KEY",
AWSRegion: "AWS_REGION",
AWSRegionAlt: "AWS_DEFAULT_REGION",
GCPProject: "GOOGLE_CLOUD_PROJECT",
}

Mutate this global at process start to swap in custom env var names — useful for multi-tenant CI runners that need to scope keys per tenant.

Errors

Source: router/router.go#L29.

var (
// ErrEmptyModel — Request.Model is empty.
ErrEmptyModel = errors.New("router: model id cannot be empty")
// ErrUnsupportedRoute — the inferred family cannot be served by
// the requested platform.
ErrUnsupportedRoute = errors.New("router: unsupported (family, platform) combination")
// ErrMissingCredentials — required credentials are missing.
// Wrapped with the field name the caller forgot.
ErrMissingCredentials = errors.New("router: missing required credentials")
// ErrNoAutoPlatform — PlatformAuto found no platform with credentials.
ErrNoAutoPlatform = errors.New("router: no platform with credentials available for the inferred family")
)

Use errors.Is to inspect:

_, err := router.Resolve(req)
switch {
case errors.Is(err, router.ErrUnsupportedRoute):
// Family doesn't run on the requested platform.
case errors.Is(err, router.ErrMissingCredentials):
// The wrapped message names the missing field.
case errors.Is(err, llmrouter.ErrInvalidConfig):
// The underlying provider's New() rejected the assembled options.
}

See also