Last updated 2026-05-17

Anthropic Provider

The anthropic provider talks to Anthropic's /v1/messages API. Unlike the OpenAI provider, which is byte-passthrough, this provider does full translation: incoming OpenAI-shaped ChatRequest values are rewritten to Anthropic's request shape, and Anthropic's SSE event stream is rewritten back into OpenAI-shaped Chunk values. The result: the same application code can target Anthropic without branching on provider, and your streaming consumer is none-the-wiser.

Import path

import (
    "github.com/elloloop/llmrouter"
    "github.com/elloloop/llmrouter/providers/anthropic"
)

Default endpoint

With no WithBaseURL override, the provider targets https://api.anthropic.com/v1. The request URL is {BaseURL}/messages.

Construction

provider, err := anthropic.New(
    llmrouter.WithAPIKey(os.Getenv("ANTHROPIC_API_KEY")),
)
if err != nil {
    log.Fatal(err)
}

The constructor wraps shared llmrouter.NewConfig, requires a non-empty API key (returns a wrapped ErrInvalidConfig otherwise), and falls back to the default base URL.

What's wire-level

The provider speaks Anthropic's /v1/messages on the wire. Two translations happen on every request:

  • Request translation. The OpenAI-shaped ChatRequest is converted to Anthropic's request body shape — system messages lifted to a top-level field, messages kept as-is, max_tokens defaulted, sampling knobs renamed where Anthropic uses different keys.
  • Response translation. Each Anthropic SSE event (message_start, content_block_delta, message_delta, message_stop) is converted to one or more OpenAI-shaped streaming chunks before being sent to the consumer.

Streaming is always on — the provider forces "stream": true in the outgoing body regardless of what the caller passed.

Request translation

The following mappings happen inside the provider:

  • System messages → top-level system. Anthropic does not accept {"role":"system"} in the messages array. The provider collects every message whose Role is "system", extracts the plain text via Message.PlainText, and joins them with "\n\n" into a single top-level system field. The system messages are then removed from the messages array.
  • max_tokens defaults to 4096. Anthropic requires max_tokens. If the caller did not set it on ChatRequest, the provider defaults to 4096 so callers don't have to know that detail.
  • Streaming is forced. stream: true is always set, even if the caller passed stream: false. Non-streaming calls are not supported in v0.1.
  • Sampling knobs pass through. temperature, top_p, top_k, and stop are copied to the outgoing body. The stop field is renamed to stop_sequences, which is what Anthropic expects. When the caller supplies ChatRequest.Raw, these knobs are read from the raw body; otherwise they are read from the typed ChatRequest fields.
  • Other fields ride on Raw. Anything Anthropic accepts that the typed ChatRequest doesn't model — tools, tool_choice, metadata — needs to be passed via ChatRequest.Raw. The provider only looks at four keys (temperature, top_p, top_k, stop) from the raw body when assembling the outgoing request; the rest is currently dropped. To get full passthrough, build the Anthropic body yourself and use the OpenAI provider against an OpenAI-compatible proxy of Anthropic instead.

Response translation

Anthropic emits a typed event stream. Each event becomes zero or more OpenAI chunks:

  • message_start → emits a role-primer chunk {"delta":{"role":"assistant"}} and captures input_tokens from the event's message.usage. The model id from message.model overrides the caller's value if it differs.
  • content_block_delta with {"type":"text_delta"} → emits a content chunk with {"delta":{"content":"..."}}. Other delta types (thinking_delta, input_json_delta) are currently dropped — see Caveats.
  • message_delta → emits a final chunk with the mapped finish_reason and the accumulated Usage object. Anthropic's stop reasons map as follows:
    • end_turn, stop_sequence → stop
    • max_tokens → length
    • tool_use → tool_calls
    • anything else → stop
  • message_stop → terminates the stream (hooks.Finish(nil)).
  • ping, content_block_start, content_block_stop → ignored. They carry no consumer-relevant payload in v0.1.

Every emitted chunk has its Object set to "chat.completion.chunk", a stable chatcmpl-<uuid> id reused across every chunk in the stream, and a Created timestamp captured at the start of the stream.

Authentication

Anthropic uses two headers, both set automatically:

  • x-api-key: <your key>
  • anthropic-version: 2023-06-01

The version header is pinned to 2023-06-01 in v0.1. A later release will allow overriding it via llmrouter.WithExtra("anthropic-version", ...) for callers that need a different API version.

Full example

package main


import (
    "context"
    "fmt"
    "log"
    "os"


    "github.com/elloloop/llmrouter"
    "github.com/elloloop/llmrouter/providers/anthropic"
)


func main() {
    p, err := anthropic.New(
        llmrouter.WithAPIKey(os.Getenv("ANTHROPIC_API_KEY")),
    )
    if err != nil {
        log.Fatal(err)
    }


    ctx := context.Background()
    stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
        Model:     "claude-3-5-sonnet-latest",
        MaxTokens: 256,
        Messages: []llmrouter.Message{
            llmrouter.TextMessage("system", "You are a concise assistant."),
            llmrouter.TextMessage("user", "Explain the speed of light in one sentence."),
        },
    })
    if err != nil {
        log.Fatal(err)
    }


    for chunk := range stream.Chunks() {
        for _, choice := range chunk.Choices {
            fmt.Print(choice.Delta.Content)
        }
    }
    if err := stream.Err(); err != nil {
        log.Fatal(err)
    }
    fmt.Println()
}

Note MaxTokens: 256. You can leave it at zero and the provider will default to 4096, but setting it explicitly is cheap insurance for cost-sensitive code.

Multiple system prompts

Pass multiple {"role":"system"} messages and the provider will join them with "\n\n" before sending. This is useful for layering a persona, a task description, and a set of constraints in separate semantically-grouped messages: 

stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model:     "claude-3-5-sonnet-latest",
    MaxTokens: 512,
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("system", "You are a senior staff engineer."),
        llmrouter.TextMessage("system", "Use markdown. Cite line numbers."),
        llmrouter.TextMessage("system", "When unsure, say so. Never invent APIs."),
        llmrouter.TextMessage("user", "Review the following Go function..."),
    },
})

The wire-level body Anthropic sees will have a single system string concatenating the three system messages.

Stop sequences

The typed ChatRequest.Stop field flows through as Anthropic's stop_sequences: 

stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model:     "claude-3-5-haiku-latest",
    MaxTokens: 200,
    Stop:      []string{"\n\nUSER:", "END_OF_RESPONSE"},
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("user", "Continue: Once upon a time"),
    },
})

When the model halts on a stop sequence, the final chunk's FinishReason is "stop" (Anthropic's stop_sequence maps to OpenAI's stop).

Vision / multimodal

Anthropic's content array shape differs from OpenAI's — image blocks use {"type":"image","source":{...}} instead of OpenAI's {"type":"image_url","image_url":...}. Because the typed surface in llmrouter.Message is OpenAI-shaped, multimodal Anthropic requests need you to either construct the Anthropic-shape content manually or pass the entire request via Raw. 

// Anthropic-shape content array, built by hand.
content, _ := json.Marshal([]map[string]any{
    {
        "type": "image",
        "source": map[string]any{
            "type":       "base64",
            "media_type": "image/jpeg",
            "data":       base64Encoded,
        },
    },
    {
        "type": "text",
        "text": "What's in this image?",
    },
})


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model:     "claude-3-5-sonnet-latest",
    MaxTokens: 512,
    Messages: []llmrouter.Message{{
        Role:    "user",
        Content: content,
    }},
})

The system-message lift (see Request translation) still applies, so you can keep using OpenAI-style system messages even when the user messages carry vision content.

Tool use

Tool use needs to ride on ChatRequest.Raw because the typed surface does not model tools or tool_choice yet. The provider currently only looks at sampling knobs in the raw body, so tool-use requests are best constructed by hand: 

reqBody := []byte(`{
  "model": "claude-3-5-sonnet-latest",
  "max_tokens": 1024,
  "tools": [
    {
      "name": "get_weather",
      "description": "Get current weather for a city",
      "input_schema": {
        "type": "object",
        "properties": {
          "city": {"type": "string"}
        },
        "required": ["city"]
      }
    }
  ],
  "messages": [
    {"role": "user", "content": "What is the weather in Paris?"}
  ]
}`)


stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "claude-3-5-sonnet-latest",
    Raw:   reqBody,
})

When the model decides to call a tool, the final chunk's FinishReason is "tool_calls" (mapped from Anthropic's tool_use).

Token counting

The provider accumulates input_tokens from message_start and output_tokens from message_delta, then attaches them to the final chunk: 

var usage *llmrouter.Usage
for chunk := range stream.Chunks() {
    for _, c := range chunk.Choices {
        fmt.Print(c.Delta.Content)
    }
    if chunk.Usage != nil {
        usage = chunk.Usage
    }
}
if usage != nil {
    fmt.Printf("\nprompt=%d completion=%d total=%d\n",
        usage.PromptTokens,
        usage.CompletionTokens,
        usage.TotalTokens,
    )
}

Caveats and known gaps

  • Chunk.Raw is the translated form. Each chunk's Raw field carries the OpenAI-shape JSON built by the provider, not the original Anthropic SSE event. If you need to inspect the original Anthropic events (e.g. to extract content_block_start with tool ids), use the official anthropic-sdk-go directly.
  • Cache metrics are not surfaced. Anthropic reports cache_creation_input_tokens and cache_read_input_tokens in message_start.usage. The provider currently only reads input_tokens and output_tokens. Use the official SDK if you need cache visibility for billing.
  • Thinking deltas are dropped. Extended-thinking models emit {"type":"thinking_delta"} events inside content_block_delta. The provider only forwards text_delta; thinking content is silently discarded. Surfacing thinking content is on the v0.3 roadmap.
  • Most raw fields are dropped. When ChatRequest.Raw is set, only the four sampling keys listed under Request translation are forwarded. This is the most common reason a tool-use request doesn't work as expected.

Error handling

Non-2xx responses surface as *llmrouter.ErrUpstream with Provider: "anthropic". Anthropic uses a typed error body shape: 

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "max_tokens: Field required"
  }
}

The same pattern as OpenAI works for branching on status:

stream, err := p.CompletionStream(ctx, req)
if err != nil {
    var upstream *llmrouter.ErrUpstream
    if errors.As(err, &upstream) {
        log.Printf("anthropic error: status=%d body=%s",
            upstream.StatusCode, upstream.Body)
        switch upstream.StatusCode {
        case 429:
            // overloaded — back off
        case 401:
            // bad api key — fail fast
        case 400:
            // malformed request
        }
        return
    }
    log.Fatal(err)
}

The body field carries up to 8 KiB of the raw response, which is enough for the typed error envelope above. Parse it with json.Unmarshal if you want to branch on error.type.

See also