Last updated 2026-05-17

OpenAI-compatible Endpoints

Many providers and self-hosted runtimes implement the OpenAI Chat Completions API on the wire — POST /chat/completions that takes an OpenAI-shaped JSON body and returns an OpenAI-shaped SSE event stream. The OpenAI provider is byte-passthrough at both ends, so pointing it at any of these endpoints Just Works: switch the base URL, switch the API key, keep everything else.

The pattern

p, err := openai.New(
    llmrouter.WithAPIKey(os.Getenv("OPENROUTER_API_KEY")),
    llmrouter.WithBaseURL("https://openrouter.ai/api/v1"),
)

That's it. The provider issues POST {BaseURL}/chat/completions with Authorization: Bearer <key>. If the upstream is OpenAI-compatible, no further translation is needed.

The rest of this page is a directory of known-compatible endpoints with their base URLs and any per-endpoint quirks. For the application-side semantics — streaming, error handling, multimodal content, token counting — see the OpenAI provider page; everything there applies here.

Hosted endpoints

OpenRouter

OpenRouter is a meta-provider — one base URL, one API key, hundreds of upstream models from dozens of vendors. 

p, err := openai.New(
    llmrouter.WithAPIKey(os.Getenv("OPENROUTER_API_KEY")),
    llmrouter.WithBaseURL("https://openrouter.ai/api/v1"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "anthropic/claude-3.5-sonnet",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Hello!"),
    },
})

OpenRouter recommends sending HTTP-Referer and X-Title headers so your traffic shows up in their dashboard with a friendly name. Both are optional. Add them via a custom HTTP transport — see the full OpenRouter example below.

Together AI

p, err := openai.New(
    llmrouter.WithAPIKey(os.Getenv("TOGETHER_API_KEY")),
    llmrouter.WithBaseURL("https://api.together.xyz/v1"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "meta-llama/Llama-3.3-70B-Instruct-Turbo",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Hello!"),
    },
})

Models are listed at together.ai/models. The full model id (with vendor prefix) is what goes in ChatRequest.Model.

Groq

Groq runs open-weight models on custom LPU silicon for very low latency. The endpoint URL has an /openai/v1 suffix that's easy to forget: 

p, err := openai.New(
    llmrouter.WithAPIKey(os.Getenv("GROQ_API_KEY")),
    llmrouter.WithBaseURL("https://api.groq.com/openai/v1"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "llama-3.3-70b-versatile",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Hello!"),
    },
})

DeepSeek

p, err := openai.New(
    llmrouter.WithAPIKey(os.Getenv("DEEPSEEK_API_KEY")),
    llmrouter.WithBaseURL("https://api.deepseek.com/v1"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "deepseek-chat",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Hello!"),
    },
})

Fireworks

p, err := openai.New(
    llmrouter.WithAPIKey(os.Getenv("FIREWORKS_API_KEY")),
    llmrouter.WithBaseURL("https://api.fireworks.ai/inference/v1"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "accounts/fireworks/models/llama-v3p3-70b-instruct",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Hello!"),
    },
})

Perplexity

Perplexity's chat completions endpoint sits at the bare /api.perplexity.ai host with no version suffix: 

p, err := openai.New(
    llmrouter.WithAPIKey(os.Getenv("PERPLEXITY_API_KEY")),
    llmrouter.WithBaseURL("https://api.perplexity.ai"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "llama-3.1-sonar-large-128k-online",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "What happened in tech today?"),
    },
})

Self-hosted runtimes

vLLM

vLLM serves any Hugging Face model with an OpenAI-compatible HTTP API. Start the server with vllm serve <model> and point the provider at port 8000: 

p, err := openai.New(
    llmrouter.WithAPIKey("none"),  // vLLM doesn't require auth by default
    llmrouter.WithBaseURL("http://your-host:8000/v1"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "meta-llama/Llama-3.1-8B-Instruct",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Hello!"),
    },
})

The Model string is the same HF model id you passed to vllm serve. WithAPIKey rejects an empty string, so pass any non-empty placeholder when the server doesn't require auth.

Ollama

Ollama runs quantized open models locally with a tiny memory footprint. It exposes an OpenAI-compatibility layer at localhost:11434/v1: 

p, err := openai.New(
    llmrouter.WithAPIKey("ollama"),  // any non-empty placeholder
    llmrouter.WithBaseURL("http://localhost:11434/v1"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "llama3.2",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Hello!"),
    },
})

LM Studio

p, err := openai.New(
    llmrouter.WithAPIKey("lm-studio"),
    llmrouter.WithBaseURL("http://localhost:1234/v1"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "lmstudio-community/Meta-Llama-3.1-8B-Instruct-GGUF",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Hello!"),
    },
})

LM Studio's GUI shows the loaded model identifier; copy that string into ChatRequest.Model.

LocalAI

p, err := openai.New(
    llmrouter.WithAPIKey("localai"),
    llmrouter.WithBaseURL("http://localhost:8080/v1"),
)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "gpt-4",  // LocalAI aliases configured in your models.yaml
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Hello!"),
    },
})

Self-hosted compatibility notes

Any HTTP server that implements POST /chat/completions with OpenAI-shaped request and SSE response bodies works. The minimum the server has to honour is:

  • Request body. An OpenAI Chat Completions JSON object on POST. The provider always sets "stream": true and (unless a caller overrides) "stream_options": {"include_usage": true}.
  • Response framing. Content-Type: text/event-stream with data: lines, each carrying an OpenAI chat.completion.chunk JSON payload, ending with a literal data: [DONE] line to terminate.
  • Error responses. Anything ≥ 400 surfaces as *llmrouter.ErrUpstream; the response body is captured (up to 1 KiB).

A server that supports the basic shape but doesn't emit a usage block on the final chunk will simply leave Chunk.Usage nil — everything else still works.

Endpoint reference

Provider Base URL Auth
OpenAIhttps://api.openai.com/v1Bearer
OpenRouterhttps://openrouter.ai/api/v1Bearer + optional HTTP-Referer / X-Title
Together AIhttps://api.together.xyz/v1Bearer
Groqhttps://api.groq.com/openai/v1Bearer
DeepSeekhttps://api.deepseek.com/v1Bearer
Fireworkshttps://api.fireworks.ai/inference/v1Bearer
Perplexityhttps://api.perplexity.aiBearer
vLLMhttp://your-host:8000/v1None / Bearer
Ollamahttp://localhost:11434/v1None
LM Studiohttp://localhost:1234/v1None
LocalAIhttp://localhost:8080/v1None / Bearer

Example: OpenRouter with custom headers

OpenRouter accepts two optional headers that improve their dashboard analytics: HTTP-Referer (your app URL) and X-Title (a human-readable app name). Wrap the HTTP client with a RoundTripper: 

package main


import (
    "context"
    "fmt"
    "log"
    "net/http"
    "os"
    "time"


    "github.com/elloloop/llmrouter"
    "github.com/elloloop/llmrouter/providers/openai"
)


type openrouterHeaders struct {
    base    http.RoundTripper
    referer string
    title   string
}


func (t *openrouterHeaders) RoundTrip(req *http.Request) (*http.Response, error) {
    if t.referer != "" {
        req.Header.Set("HTTP-Referer", t.referer)
    }
    if t.title != "" {
        req.Header.Set("X-Title", t.title)
    }
    return t.base.RoundTrip(req)
}


func main() {
    httpClient := &http.Client{
        Timeout: 120 * time.Second,
        Transport: &openrouterHeaders{
            base:    http.DefaultTransport,
            referer: "https://my-app.example.com",
            title:   "My Demo App",
        },
    }


    p, err := openai.New(
        llmrouter.WithAPIKey(os.Getenv("OPENROUTER_API_KEY")),
        llmrouter.WithBaseURL("https://openrouter.ai/api/v1"),
        llmrouter.WithHTTPClient(httpClient),
    )
    if err != nil {
        log.Fatal(err)
    }


    stream, err := p.CompletionStream(context.Background(), llmrouter.ChatRequest{
        Model: "anthropic/claude-3.5-sonnet",
        Messages: []llmrouter.Message{
            llmrouter.TextMessage("user", "What's the capital of France?"),
        },
    })
    if err != nil {
        log.Fatal(err)
    }


    for chunk := range stream.Chunks() {
        for _, c := range chunk.Choices {
            fmt.Print(c.Delta.Content)
        }
    }
    if err := stream.Err(); err != nil {
        log.Fatal(err)
    }
    fmt.Println()
}

Example: Ollama for local dev

For local development you typically don't want a real API key in code paths under test. Point at Ollama, pass a placeholder key, and your code is unchanged from the production OpenAI path: 

package main


import (
    "context"
    "fmt"
    "log"
    "os"


    "github.com/elloloop/llmrouter"
    "github.com/elloloop/llmrouter/providers/openai"
)


func main() {
    // Pick the upstream from env. Default to OpenAI; override with
    // OLLAMA_HOST=http://localhost:11434/v1 when developing offline.
    baseURL := "https://api.openai.com/v1"
    apiKey := os.Getenv("OPENAI_API_KEY")
    model := "gpt-4o-mini"


    if host := os.Getenv("OLLAMA_HOST"); host != "" {
        baseURL = host
        apiKey = "ollama"     // non-empty placeholder
        model = "llama3.2"
    }


    p, err := openai.New(
        llmrouter.WithAPIKey(apiKey),
        llmrouter.WithBaseURL(baseURL),
    )
    if err != nil {
        log.Fatal(err)
    }


    stream, err := p.CompletionStream(context.Background(), llmrouter.ChatRequest{
        Model: model,
        Messages: []llmrouter.Message{
            llmrouter.TextMessage("user", "One-line summary of CRDTs?"),
        },
    })
    if err != nil {
        log.Fatal(err)
    }
    for chunk := range stream.Chunks() {
        for _, c := range chunk.Choices {
            fmt.Print(c.Delta.Content)
        }
    }
    if err := stream.Err(); err != nil {
        log.Fatal(err)
    }
    fmt.Println()
}

Provider-agnostic factory

For code that swaps providers based on configuration, a tiny factory keeps the call sites clean. Both providers satisfy the llmrouter.Provider interface: 

package router


import (
    "fmt"
    "os"


    "github.com/elloloop/llmrouter"
    "github.com/elloloop/llmrouter/providers/anthropic"
    "github.com/elloloop/llmrouter/providers/openai"
)


// Build returns a Provider keyed by a short label. Add cases as you
// adopt more endpoints; the call site never branches on provider.
func Build(label string) (llmrouter.Provider, error) {
    switch label {
    case "openai":
        return openai.New(llmrouter.WithAPIKey(os.Getenv("OPENAI_API_KEY")))


    case "anthropic":
        return anthropic.New(llmrouter.WithAPIKey(os.Getenv("ANTHROPIC_API_KEY")))


    case "openrouter":
        return openai.New(
            llmrouter.WithAPIKey(os.Getenv("OPENROUTER_API_KEY")),
            llmrouter.WithBaseURL("https://openrouter.ai/api/v1"),
        )


    case "groq":
        return openai.New(
            llmrouter.WithAPIKey(os.Getenv("GROQ_API_KEY")),
            llmrouter.WithBaseURL("https://api.groq.com/openai/v1"),
        )


    case "ollama":
        return openai.New(
            llmrouter.WithAPIKey("ollama"),
            llmrouter.WithBaseURL("http://localhost:11434/v1"),
        )


    default:
        return nil, fmt.Errorf("unknown provider: %q", label)
    }
}

Call sites pass llmrouter.Provider around and use only its two methods (Name and CompletionStream). Swapping the upstream is a config change, not a code change.

See also