Last updated 2026-05-18

What It Is

Rate Limiter is a reusable enforcement engine for quotas, rate limits, and active concurrency. It accepts the limits a caller already decided should apply and enforces them atomically in Redis.

Caller responsibilities

• Know the business rules.
• Know the user, organization, workspace, or provider context.
• Convert plans and contracts into concrete Limit messages.
• Pass those limits on every request.

Service responsibilities

• Validate supplied limits.
• Evaluate algorithms and windows.
• Read and mutate Redis counters atomically.
• Return decisions, usage state, retry hints, and debug explanations.
• Emit append-only usage events when configured.

Opaque fields

scope_key, action, and unit are opaque strings. Examples include user:user_123, assistant.llm.tokens, and tokens. The service stores and returns them but does not interpret their business meaning.