Last updated 2026-05-17

Embeddings

Embeddings turn text into fixed-size float vectors that capture semantic similarity. They are the bedrock of retrieval-augmented generation, semantic search, clustering, and classification. In llmrouter, embeddings live behind a separate interface from chat completions because the request and response shapes are fundamentally different: there is no streaming, no Stream handle, no SSE — just a single round-trip that returns one vector per input.

The Embedder interface

Every embedding-capable provider implements Embedder. Like Provider, it has exactly one method, and implementations are concurrency-safe:

type Embedder interface {
    Embed(ctx context.Context, req EmbedRequest) (*EmbedResponse, error)
}

Not every provider is an Embedder. Anthropic, Groq, OpenRouter, Fireworks, DeepSeek, xAI, Perplexity, and Cerebras have no first-party embedding endpoint — for those vendors, llmrouter recommends pairing the chat provider with a dedicated embedding provider such as Voyage AI (the canonical pairing for Anthropic Claude users) or OpenAI.

EmbedRequest

The polymorphic request shape. Vendor-specific extras travel through Raw (full byte passthrough) or via the typed fields below.

type EmbedRequest struct {
    Model          string          // e.g. "text-embedding-3-small", "voyage-3-large"
    Inputs         []string        // texts to embed; index-aligned with the response
    Dimensions     int             // optional; lower-dim output for OpenAI v3 + others
    TaskType       string          // task hint, normalised cross-vendor (see table below)
    EncodingFormat string          // "float" (default) or "base64"
    User           string          // OpenAI-style end-user id for telemetry
    Raw            json.RawMessage // optional byte passthrough; Model is overlaid
}

Field-by-field

Model: The model identifier. The literal string is sent to the upstream; providers don't try to be clever about model aliases. Common values: text-embedding-3-small, text-embedding-3-large (OpenAI / Azure), embed-english-v3.0 (Cohere), voyage-3, voyage-3-large (Voyage), text-embedding-005 (Vertex / Gemini), mistral-embed (Mistral), amazon.titan-embed-text-v2:0 (Bedrock).
Inputs: The list of strings to embed. EmbedResponse.Embeddings is index-aligned with this slice: the vector at position i corresponds to Inputs[i]. Most providers accept arrays; legacy single-string endpoints batch client-side inside the provider.
Dimensions: Optional. Requests a lower-dimensional output. Supported by OpenAI text-embedding-3-* (256 to 3072), Voyage (256 / 512 / 1024 / 2048), and a handful of others. Zero means "use the model's default dimension."
TaskType: Optional task hint. llmrouter uses the Gemini/Vertex/Voyage vocabulary as the canonical form and translates per-vendor. See cross-vendor task-type mapping below.
EncodingFormat: "float" (default) returns []float32 per input. "base64" returns the base64 wire encoding; the library still decodes it to []float32 in the response — the field only controls the wire format. Providers that support only one format ignore this field.
User: OpenAI-style end-user identifier. Used for abuse tracking and telemetry. Forwarded to OpenAI and Azure; ignored by others.
Raw: Byte passthrough escape hatch. If non-nil, this JSON object is the outgoing request body with Model overlaid. Use Raw for vendor-specific fields the typed surface doesn't model — for example, Cohere's input_type and embedding_types, Voyage's truncation, Bedrock Titan's normalize.

EmbedResponse

type EmbedResponse struct {
    Model      string          // resolved model id echoed by the provider
    Embeddings [][]float32     // index-aligned with EmbedRequest.Inputs
    Usage      *Usage          // PromptTokens / TotalTokens when available
    Raw        json.RawMessage // original wire-format JSON
}

Model: The model id the upstream actually used. For some providers this echoes the request verbatim; for others it includes a version suffix (e.g. text-embedding-3-small-2024). Useful for logging which dimension space your vectors live in.
Embeddings: The vectors, one per input. len(Embeddings) == len(req.Inputs) always — the library reorders if the provider returns out-of-order results (Cohere is index-tagged on the wire).
Usage: Token usage. Embedding providers populate PromptTokens (sometimes also TotalTokens). May be nil for vendors that don't report usage.
Raw: Original wire bytes for callers that want to forward the response without re-marshaling.

Cross-vendor task-type mapping

Different vendors spell the same task hint differently. The library accepts the Vertex/Gemini vocabulary as canonical and translates per-provider on the way out. Pass the canonical value in TaskType; the provider rewrites it for you.

Canonical (Vertex / Gemini / library)	Cohere `input_type`	Voyage `input_type`	OpenAI / Mistral
`RETRIEVAL_QUERY`	`search_query`	`query`	(not modeled — ignored)
`RETRIEVAL_DOCUMENT`	`search_document`	`document`	(ignored)
`SEMANTIC_SIMILARITY`	`search_document`	`document`	(ignored)
`CLASSIFICATION`	`classification`	`document`	(ignored)
`CLUSTERING`	`clustering`	`document`	(ignored)
`QUESTION_ANSWERING`	`search_query`	`query`	(ignored)

If you'd rather speak each vendor's native vocabulary, set TaskType to the empty string and pass the vendor's raw field via Raw.

Full Go example

A complete program that embeds three strings against OpenAI and prints the cosine similarity between each pair.

package main

import (
    "context"
    "fmt"
    "log"
    "math"
    "os"

    "github.com/elloloop/llmrouter"
    "github.com/elloloop/llmrouter/providers/openai"
)

func main() {
    p, err := openai.New(llmrouter.WithAPIKey(os.Getenv("OPENAI_API_KEY")))
    if err != nil {
        log.Fatal(err)
    }

    inputs := []string{
        "The cat sat on the mat.",
        "A feline rested on the rug.",
        "Quantum chromodynamics is hard.",
    }

    resp, err := p.Embed(context.Background(), llmrouter.EmbedRequest{
        Model:      "text-embedding-3-small",
        Inputs:     inputs,
        Dimensions: 256,
        TaskType:   "SEMANTIC_SIMILARITY",
    })
    if err != nil {
        log.Fatal(err)
    }

    for i := 0; i < len(inputs); i++ {
        for j := i + 1; j < len(inputs); j++ {
            sim := cosine(resp.Embeddings[i], resp.Embeddings[j])
            fmt.Printf("sim(%d,%d) = %.3f\n", i, j, sim)
        }
    }
    if resp.Usage != nil {
        fmt.Printf("prompt_tokens=%d total_tokens=%d\n",
            resp.Usage.PromptTokens, resp.Usage.TotalTokens)
    }
}

func cosine(a, b []float32) float32 {
    var dot, na, nb float64
    for i := range a {
        dot += float64(a[i]) * float64(b[i])
        na += float64(a[i]) * float64(a[i])
        nb += float64(b[i]) * float64(b[i])
    }
    return float32(dot / (math.Sqrt(na) * math.Sqrt(nb)))
}

Raw passthrough

For vendor-specific fields not on the typed surface, build the JSON yourself and pass it as Raw. The library overlays Model so you can swap models without rewriting the body. Cohere's embedding_types is a good example:

raw := []byte(`{
  "input": ["hello", "world"],
  "input_type": "search_document",
  "embedding_types": ["float", "int8"]
}`)

resp, err := cohereProvider.Embed(ctx, llmrouter.EmbedRequest{
    Model: "embed-english-v3.0",
    Raw:   raw,
})

The typed Embeddings field in the response always contains the float vectors; richer wire formats are available on Raw.