Last updated 2026-05-17

Package llmrouter — Rerank

Every public symbol declared in rerank.go. One interface — Reranker — plus the request and response types it speaks.

See the Rerank concept page for when and why to use this surface in a RAG pipeline.

Import path:

import "github.com/elloloop/llmrouter"

Reranker

The rerank contract. Implementations are concurrency-safe.

type Reranker interface {
    Name() string
    Rerank(ctx context.Context, req RerankRequest) (*RerankResponse, error)
}

Rerank is a single-round-trip call — there is no streaming, no Stream handle. A non-nil error is returned on synchronous failures (bad config, non-2xx HTTP status, malformed response body). Name returns the provider's short id, matching the value also returned by the same provider's Provider.Name().

RerankRequest

The rerank-request shape. Vendor-agnostic.

type RerankRequest struct {
    Model     string          // vendor model id; e.g. "rerank-v3.5"
    Query     string          // the user's question
    Documents []string        // candidates to rerank
    TopN      int             // soft cap on returned results; 0 means "all"
    Raw       json.RawMessage // byte passthrough; typed fields overlay
}

Model: The model identifier. The literal string is sent to the upstream; providers don't alias. Common values: rerank-v3.5 (Cohere), rerank-2 (Voyage), Salesforce/Llama-Rank-V1 (Together).
Query: The query string each document is scored against. Required. Length limits are vendor-specific — most accept several kilobytes.
Documents: The candidate documents. Required. Order is preserved on the wire; results refer back via RerankResult.Index. Length limits per document vary by vendor; Cohere allows ~4 KB, Voyage similar, Together shorter. Truncate aggressively — long documents cost more and rarely improve quality past the first ~1 KB.
TopN: Soft cap on returned results. 0 means “return everything ranked.” The vendor may return fewer than TopN if its internal score floor isn't met. Common values: 5-10 for chat RAG, 15-25 for citation use cases.
Raw: Byte passthrough escape hatch. Vendor-specific fields (Cohere rank_fields, Voyage truncation) can be sent through Raw. The typed fields above overlay Raw after marshalling, so setting both is safe — the typed values win.

RerankResponse

The rerank-response shape.

type RerankResponse struct {
    Results []RerankResult  // sorted by descending Score
    Usage   *Usage          // tokens billed (when reported)
    Raw     json.RawMessage // original wire JSON
}

Results: The reranked results, sorted by descending Score. Always non-nil but may be empty if every candidate scored below the vendor's internal floor.
Usage: Token / billable-unit counts when the vendor reports them. Cohere and Voyage both report; Together does not always. nil when no usage info was returned.
Raw: Original wire JSON for byte-level passthrough or to read vendor-specific fields the typed surface omits.

RerankResult

A single reranked result.

type RerankResult struct {
    Index    int     // position in the original RerankRequest.Documents slice
    Score    float64 // 0.0-1.0; higher = more relevant
    Document string  // copy of the document text, for convenience
}

Index: The position in the original RerankRequest.Documents slice. Use this to join back to metadata you kept alongside the documents (chunk id, source URL, page number, ...). Never reordered.
Score: The relevance score in [0.0, 1.0]. Vendor-specific calibration — a score of 0.8 from Cohere is not directly comparable to 0.8 from Voyage. Pick one provider and calibrate thresholds against your own evaluation set.
Document: A copy of the document text from the request, included for logging / debugging convenience. The library returns this verbatim from the request — the upstream does not echo it back on every vendor.

Implementations

Reranker is implemented today by:

Cohere — rerank-v3.5, rerank-english-v3.0, rerank-multilingual-v3.0.
Voyage — rerank-2, rerank-2-lite.
Together via the OpenAI-compatible provider with WithBaseURL("https://api.together.xyz/v1") — Salesforce/Llama-Rank-V1.