Last updated 2026-05-17

Configuration & Options

Every provider in llmrouter is constructed with the same shape: a variadic list of functional options. 

p, err := openai.New(
    llmrouter.WithAPIKey("sk-..."),
    llmrouter.WithBaseURL("https://openrouter.ai/api/v1"),
    llmrouter.WithTimeout(30*time.Second),
)

The options live in llmrouter/options.go and are shared across every provider. A provider's New(opts ...llmrouter.Option) walks them, builds a *llmrouter.Config, then validates the bits it cares about (API key, base URL).

Why functional options

The alternative — a public Config struct passed to New(cfg Config) — has three problems this design avoids:

  • Zero-value safety. A struct's zero value is always valid Go, but it's almost never a valid configuration. A caller who forgets to set APIKey sees an upstream 401 at request time, not a clear error at construction time. Functional options can require WithAPIKey and tell you why New failed.
  • Forward-compatibility. Adding a new field to a struct doesn't break callers, but adding a required field does. Adding a new WithX function is purely additive — old code keeps compiling and old behavior is preserved.
  • Per-option validation. Each option returns an error. The first failing option short-circuits New and gives you a precise pointer to which option was wrong, with a message that mentions that option.

The Option type

type Option func(*Config) error

That's it. An Option is a function that mutates a Config and reports whether the mutation succeeded. Internally NewConfig walks the slice: 

func NewConfig(opts ...Option) (*Config, error) {
    c := &Config{Timeout: 120 * time.Second}
    for _, opt := range opts {
        if opt == nil {
            continue
        }
        if err := opt(c); err != nil {
            return nil, err
        }
    }
    return c, nil
}

nil options are skipped silently — this matters when you build the option list conditionally: 

var baseURL llmrouter.Option
if env := os.Getenv("LLM_BASE_URL"); env != "" {
    baseURL = llmrouter.WithBaseURL(env)
}
p, err := openai.New(
    llmrouter.WithAPIKey(os.Getenv("OPENAI_API_KEY")),
    baseURL, // nil here is fine; falls back to provider default
)

Available options

WithAPIKey(key string)

Sets the API key. The string is trimmed; empty (after trim) is an error. Every provider requires this — New wraps llmrouter.ErrInvalidConfig if it's missing. 

llmrouter.WithAPIKey(os.Getenv("OPENAI_API_KEY"))

WithBaseURL(url string)

Overrides the provider's default base URL. Validated as a URL; trailing slashes are stripped. Most useful for the OpenAI provider pointed at an OpenAI-compatible endpoint:

EndpointBase URL
OpenAI (default) https://api.openai.com/v1
OpenRouter https://openrouter.ai/api/v1
Together https://api.together.xyz/v1
Groq https://api.groq.com/openai/v1
Fireworks https://api.fireworks.ai/inference/v1
vLLM (self-hosted) http://localhost:8000/v1
Ollama http://localhost:11434/v1
Anthropic (default) https://api.anthropic.com/v1

For OpenAI-compatible endpoints that don't require a real key (vLLM, Ollama in default config), you still must pass WithAPIKey with something non-empty — the OpenAI provider always sets the Authorization header. Pass any placeholder: 

p, err := openai.New(
    llmrouter.WithAPIKey("ollama"),
    llmrouter.WithBaseURL("http://localhost:11434/v1"),
)

WithHTTPClient(client *http.Client)

Supplies a custom *http.Client. Useful for wrapping the transport with retries, OpenTelemetry instrumentation, proxying, mTLS, or anything else that lives at the HTTP layer. Required when you want behavior WithTimeout can't express.

When you pass this, WithTimeout is ignored. Your client's transport governs all timeouts. Set them on the client yourself. 

package main


import (
    "errors"
    "fmt"
    "log"
    "net/http"
    "os"
    "time"


    "github.com/elloloop/llmrouter"
    "github.com/elloloop/llmrouter/providers/openai"
)


// retryingTransport retries on 429 and 5xx responses up to maxAttempts
// times, with exponential backoff. It does NOT retry on 4xx (other
// than 429) or network errors — those need application-level decisions.
type retryingTransport struct {
    base        http.RoundTripper
    maxAttempts int
}


func (t *retryingTransport) RoundTrip(req *http.Request) (*http.Response, error) {
    var lastResp *http.Response
    var lastErr error
    for attempt := 0; attempt < t.maxAttempts; attempt++ {
        if attempt > 0 {
            select {
            case <-time.After(backoff(attempt)):
            case <-req.Context().Done():
                return nil, req.Context().Err()
            }
        }
        resp, err := t.base.RoundTrip(req)
        if err != nil {
            return nil, err
        }
        if resp.StatusCode != 429 && resp.StatusCode < 500 {
            return resp, nil
        }
        lastResp = resp
        if attempt+1 < t.maxAttempts {
            resp.Body.Close()
        }
        lastErr = errors.New(resp.Status)
    }
    if lastResp != nil {
        return lastResp, nil
    }
    return nil, fmt.Errorf("retries exhausted: %w", lastErr)
}


func backoff(attempt int) time.Duration {
    return time.Duration(1<<attempt) * time.Second
}


func main() {
    client := &http.Client{
        Timeout: 2 * time.Minute,
        Transport: &retryingTransport{
            base:        http.DefaultTransport,
            maxAttempts: 3,
        },
    }
    p, err := openai.New(
        llmrouter.WithAPIKey(os.Getenv("OPENAI_API_KEY")),
        llmrouter.WithHTTPClient(client),
    )
    if err != nil {
        log.Fatal(err)
    }
    _ = p // use p ...
}

WithTimeout(d time.Duration)

Sets the per-request HTTP client timeout used by the default client. Default is 120 seconds. Ignored if WithHTTPClient is also supplied. 

llmrouter.WithTimeout(30 * time.Second)

WithTimeout vs context.WithTimeout

These do different things and you usually want both:

  • WithTimeout is a setting on the HTTP client, applied at provider-construction time. It governs the entire HTTP request lifecycle — connect, headers, body. It kicks in even if the caller forgets to pass a deadline.
  • context.WithTimeout is a per-call deadline. It governs this request, and it propagates through the stream — cancelling the context cancels the producer goroutine and the upstream HTTP read.

Use both for defense in depth: WithTimeout as the upper bound the provider will ever wait, WithTimeout on the context as the request-specific budget. 

// Provider-level safety net.
p, _ := openai.New(
    llmrouter.WithAPIKey(os.Getenv("OPENAI_API_KEY")),
    llmrouter.WithTimeout(2*time.Minute), // hard ceiling
)


// Per-call deadline.
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()


stream, err := p.CompletionStream(ctx, req)

WithExtra(key string, value any)

Stores a provider-specific value on the Config. Providers document the keys they read. 

// AWS Bedrock (planned):
llmrouter.WithExtra("region", "us-east-1")
llmrouter.WithExtra("aws_profile", "production")


// Azure OpenAI (planned):
llmrouter.WithExtra("api-version", "2024-10-21")
llmrouter.WithExtra("deployment", "gpt-4o-prod")


// Vertex AI (planned):
llmrouter.WithExtra("project", "my-gcp-project")
llmrouter.WithExtra("region", "us-central1")

Empty keys are an error. Values are typed as any; providers cast or assert on read. Unknown keys are ignored by providers that don't care about them, so it's safe to pass a superset of options when you have provider-selecting code.

Validation

Options validate eagerly. NewConfig walks them in order and returns the first failing option's error. The caller's New wraps that error with llmrouter.ErrInvalidConfig for sentinel matching:

providers/openai/openai.go
func New(opts ...llmrouter.Option) (*Provider, error) {
    cfg, err := llmrouter.NewConfig(opts...)
    if err != nil {
        return nil, err
    }
    if cfg.APIKey == "" {
        return nil, fmt.Errorf("%w: api key required", llmrouter.ErrInvalidConfig)
    }
    if cfg.BaseURL == "" {
        cfg.BaseURL = defaultBaseURL
    }
    return &Provider{cfg: cfg}, nil
}

So you can detect configuration errors specifically:

p, err := openai.New(llmrouter.WithAPIKey(""))
if errors.Is(err, llmrouter.ErrInvalidConfig) {
    log.Fatalf("misconfigured: %v", err)
}

See Error Handling for the full error taxonomy.

Full example: a provider factory

Real applications usually pick a provider at runtime from configuration. The factory below builds either OpenAI or Anthropic from a config struct, with sensible per-provider defaults: 

package main


import (
    "context"
    "errors"
    "fmt"
    "log"
    "net/http"
    "os"
    "time"


    "github.com/elloloop/llmrouter"
    "github.com/elloloop/llmrouter/providers/anthropic"
    "github.com/elloloop/llmrouter/providers/openai"
)


// ProviderConfig is the application's view of provider settings. Real
// code would populate this from flags / env / config file.
type ProviderConfig struct {
    Name    string        // "openai", "anthropic", "openrouter"
    APIKey  string
    BaseURL string        // optional override (mostly for OpenAI-compatible)
    Timeout time.Duration // 0 = default 120s
}


// NewProvider builds an llmrouter.Provider from a ProviderConfig. It
// returns ErrInvalidConfig for misconfiguration so callers can decide
// whether to fail-fast or fall back.
func NewProvider(cfg ProviderConfig) (llmrouter.Provider, error) {
    if cfg.APIKey == "" {
        return nil, fmt.Errorf("%w: api key required for %s",
            llmrouter.ErrInvalidConfig, cfg.Name)
    }
    opts := []llmrouter.Option{llmrouter.WithAPIKey(cfg.APIKey)}
    if cfg.BaseURL != "" {
        opts = append(opts, llmrouter.WithBaseURL(cfg.BaseURL))
    }
    if cfg.Timeout > 0 {
        opts = append(opts, llmrouter.WithTimeout(cfg.Timeout))
    }


    switch cfg.Name {
    case "openai":
        return openai.New(opts...)
    case "anthropic":
        return anthropic.New(opts...)
    case "openrouter":
        opts = append(opts, llmrouter.WithBaseURL("https://openrouter.ai/api/v1"))
        return openai.New(opts...)
    case "groq":
        opts = append(opts, llmrouter.WithBaseURL("https://api.groq.com/openai/v1"))
        return openai.New(opts...)
    default:
        return nil, fmt.Errorf("%w: unknown provider %q",
            llmrouter.ErrInvalidConfig, cfg.Name)
    }
}


func main() {
    cfg := ProviderConfig{
        Name:    os.Getenv("LLM_PROVIDER"),
        APIKey:  os.Getenv("LLM_API_KEY"),
        Timeout: 60 * time.Second,
    }
    p, err := NewProvider(cfg)
    if err != nil {
        if errors.Is(err, llmrouter.ErrInvalidConfig) {
            log.Fatalf("invalid config: %v", err)
        }
        log.Fatal(err)
    }
    log.Printf("using provider: %s", p.Name())


    ctx := context.Background()
    stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
        Model: "gpt-4o-mini",
        Messages: []llmrouter.Message{
            llmrouter.TextMessage("user", "hi"),
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    for chunk := range stream.Chunks() {
        for _, c := range chunk.Choices {
            fmt.Print(c.Delta.Content)
        }
    }
    fmt.Println()
    if err := stream.Err(); err != nil {
        log.Fatal(err)
    }
}


// Suppress unused import in this trimmed example.
var _ = http.DefaultClient

Next steps