Last updated 2026-05-17

Package openai

The OpenAI provider, plus any OpenAI-API-compatible upstream (Together, Groq, OpenRouter, Azure OpenAI, self-hosted vLLM, …) reached via llmrouter.WithBaseURL. Streaming-only in v0.1. Chunks preserve the raw wire-format JSON so passthrough proxies can forward bytes without re-marshaling. Source: providers/openai/openai.go.

Import path: 

import "github.com/elloloop/llmrouter/providers/openai"

Constants

const defaultBaseURL = "https://api.openai.com/v1"

Unexported. Used when the caller does not supply WithBaseURL. Source: openai.go#L23.

Provider

Implements llmrouter.Provider against the OpenAI Chat Completions wire format. Source: openai.go#L27. 

type Provider struct {
    // unexported: cfg *llmrouter.Config
}

New

func New(opts ...llmrouter.Option) (*Provider, error)

Constructs an OpenAI provider. Source: openai.go#L33.

Behavior:

  • Calls llmrouter.NewConfig(opts...) and surfaces the first option error.
  • Required: WithAPIKey. Returns fmt.Errorf("%w: api key required", llmrouter.ErrInvalidConfig) if not supplied.
  • Default BaseURL: https://api.openai.com/v1 when WithBaseURL is not supplied.

Example: stock OpenAI

p, err := openai.New(llmrouter.WithAPIKey(os.Getenv("OPENAI_API_KEY")))
if err != nil {
    log.Fatal(err)
}

Example: OpenRouter via base URL

p, err := openai.New(
    llmrouter.WithAPIKey(os.Getenv("OPENROUTER_API_KEY")),
    llmrouter.WithBaseURL("https://openrouter.ai/api/v1"),
)

Provider.Name

func (p *Provider) Name() string

Returns the literal string "openai". The value does not change when the base URL is overridden — Groq, Together, OpenRouter, and self-hosted endpoints all still report "openai". Source: openai.go#L48.

Provider.CompletionStream

func (p *Provider) CompletionStream(ctx context.Context, req llmrouter.ChatRequest) (*llmrouter.Stream, error)

Opens a streaming chat completion against the configured base URL and returns a *llmrouter.Stream yielding normalized chunks. Source: openai.go#L53.

Request body

Assembled by buildRequestBody:

  • Raw passthrough. If req.Raw is non-empty, the bytes are unmarshaled into a map[string]json.RawMessage. Any field already present is preserved (tools, vision, response_format, …). Returns fmt.Errorf("openai: invalid raw request: %w", err) on malformed JSON.
  • Typed marshaling. If req.Raw is empty, the ChatRequest struct is marshaled (with Stream pre-set to true) and then unmarshaled back into a map so the same overlay logic can run.
  • Model overlay. If req.Model is non-empty, the marshaled string overwrites any existing "model" key. (So Raw bytes' model is replaced whenever req.Model is set.)
  • stream is forced to true.
  • stream_options is set to {"include_usage": true} unless already present in the raw map, ensuring the upstream emits a final usage block.

HTTP call

  • Method: POST
  • URL: <BaseURL>/chat/completions
  • Headers:
    • Content-Type: application/json
    • Accept: text/event-stream
    • Authorization: Bearer <api-key>
  • HTTP client is p.cfg.HTTP() (lazy default, 120 s timeout, unless WithHTTPClient supplied one).
  • On HTTP ≥ 400, reads up to 1 KiB of the body (trimmed), closes the response, and returns a *llmrouter.ErrUpstream with Provider: "openai".

SSE pump

On success, spawns pumpSSE in a goroutine and returns a *llmrouter.Stream immediately. The pump:

  • Reads the response body line by line via bufio.Scanner (initial buffer 64 KiB, max 1 MiB).
  • Accumulates data: and data: lines into a dataLines slice. Other prefixes — event:, id:, retry:, comments (:), heartbeats — are silently dropped.
  • On a blank line, joins dataLines with "\n" (multi-line data: support per the SSE spec) and dispatches:
    • [DONE] → hooks.Finish(nil), return.
    • Anything else → decodeChunk. Malformed JSON does NOT abort the stream — the payload is skipped and scanning continues. This protects against OpenAI-compatible proxies that emit non-standard keepalive payloads.
  • Before each scan iteration, checks ctx.Done() and calls hooks.Finish(ctx.Err()) on cancel.
  • If hooks.Send returns false (consumer cancelled), calls hooks.Finish(ctx.Err()) and returns.
  • On scanner error other than io.EOF, calls hooks.Finish(fmt.Errorf("openai: read stream: %w", err)).
  • On clean EOF without an explicit [DONE], calls hooks.Finish(nil).
  • The response body is closed via defer in every exit path.

Chunk decoding

decodeChunk unmarshals each payload into a typed wire struct, copies the fields into llmrouter.Chunk, and sets Chunk.Raw to json.RawMessage(payload) — the original bytes from the SSE stream. Usage is populated only when present in the payload (typically the final chunk when stream_options.include_usage is honored).

Example: minimal completion

p, err := openai.New(llmrouter.WithAPIKey(os.Getenv("OPENAI_API_KEY")))
if err != nil {
    log.Fatal(err)
}


stream, err := p.CompletionStream(context.Background(), llmrouter.ChatRequest{
    Model: "gpt-4o-mini",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "List three primary colours."),
    },
})
if err != nil {
    log.Fatal(err)
}


for chunk := range stream.Chunks() {
    for _, c := range chunk.Choices {
        fmt.Print(c.Delta.Content)
    }
}
if err := stream.Err(); err != nil {
    log.Fatal(err)
}

Example: forwarding raw bytes through a proxy

// Inbound HTTP handler that re-emits OpenAI chunks verbatim downstream.
func proxy(w http.ResponseWriter, r *http.Request) {
    body, _ := io.ReadAll(r.Body)
    stream, err := provider.CompletionStream(r.Context(), llmrouter.ChatRequest{Raw: body})
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadGateway)
        return
    }
    w.Header().Set("Content-Type", "text/event-stream")
    flusher := w.(http.Flusher)
    for chunk := range stream.Chunks() {
        fmt.Fprintf(w, "data: %s\n\n", chunk.Raw)
        flusher.Flush()
    }
    fmt.Fprint(w, "data: [DONE]\n\n")
}

Example: tool-calls via raw passthrough

raw := []byte(`{
  "model": "gpt-4o-mini",
  "messages": [{"role":"user","content":"What is the weather in Paris?"}],
  "tools": [{
    "type":"function",
    "function":{"name":"get_weather","parameters":{"type":"object","properties":{"city":{"type":"string"}}}}
  }],
  "tool_choice": "auto"
}`)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{Raw: raw})
// stream.Chunks() yields delta chunks; tool calls arrive in the raw
// bytes (Chunk.Raw) — inspect them directly if you need tool support.