Last updated 2026-05-17

Cohere Provider

The cohere provider implements Provider (chat via the Command family) and Embedder (Embed v3 family). No TTS, no STT. Cohere's embedding models are particularly strong for multilingual retrieval and offer asymmetric query/document encoding.

Import path

import (
    "github.com/elloloop/llmrouter"
    "github.com/elloloop/llmrouter/providers/cohere"
)

Construction

p, err := cohere.New(
    llmrouter.WithAPIKey(os.Getenv("COHERE_API_KEY")),
)

Default endpoint: https://api.cohere.com/v2. Auth is via Authorization: Bearer <key>.

Chat models

  • command-r-plus — flagship, 128K context, tool use.
  • command-r — balanced.
  • command-r7b — small, fast, on-prem-friendly.
  • command-light — legacy lightweight.

Chat example

stream, err := p.CompletionStream(ctx, llmrouter.ChatRequest{
    Model: "command-r-plus",
    Messages: []llmrouter.Message{
        llmrouter.TextMessage("system", "You are concise."),
        llmrouter.TextMessage("user", "What is RAG?"),
    },
})


for chunk := range stream.Chunks() {
    for _, c := range chunk.Choices {
        fmt.Print(c.Delta.Content)
    }
}

The provider translates the OpenAI messages array into Cohere's v2 /chat body (Cohere v2 also uses messages, but with a slightly different content-block shape that the library rewrites). The SSE stream is translated back to llmrouter.Chunk.

Embeddings

  • embed-english-v3.0 — 1024 dims, English.
  • embed-multilingual-v3.0 — 1024 dims, 100+ languages.
  • embed-english-light-v3.0 — 384 dims, cheaper.
  • embed-multilingual-light-v3.0 — 384 dims, cheaper.
resp, err := p.Embed(ctx, llmrouter.EmbedRequest{
    Model:    "embed-english-v3.0",
    Inputs:   []string{"What is RAG?"},
    TaskType: "RETRIEVAL_QUERY", // → Cohere input_type "search_query"
})

input_type mapping

Cohere's input_type is mandatory on v3 models. The library derives it from TaskType:

Canonical TaskType Cohere input_type
RETRIEVAL_QUERYsearch_query
RETRIEVAL_DOCUMENTsearch_document
CLASSIFICATIONclassification
CLUSTERINGclustering

When TaskType is empty, the provider defaults to search_document to match Cohere's most common use case.

Error handling

Non-2xx responses surface as *llmrouter.ErrUpstream with Provider == "cohere".

Caveats

  • Trial vs. production keys. Trial keys are heavily rate-limited; production keys live in the dashboard under "API keys" → "Production."
  • input_type is required. Sending an embedding request without it (i.e. with no TaskType) used to 400; the library now defaults to search_document.

See also