Documentation v0.4.0
Home GitHub

Architecture

Understand how go-llm-router turns a provider name into a unified Agent and normalizes provider responses.

System overview

graph TB
    App[Caller Application]
    Server[cmd/test]
    Router[core/router]
    Contract[core Agent / StreamAgent]
    Policy[Shared Policy Reasoning / Mode]
    Providers[Provider Adapters]
    APIs[Provider APIs]
    OAuth[core/oauth]
    Keychain[System Keychain]

    App --> Router
    App --> Contract
    Server --> Router
    Router --> Contract
    Contract --> Policy
    Contract --> Providers
    Providers --> APIs
    OAuth --> Keychain
    OAuth --> Providers

Layers

Layer Packages Responsibility
Caller Application code, cmd/test Build messages and tools; choose model, reasoning level, execution mode, and synchronous or streaming consumption
Router core/router Parse a provider name and construct the matching core.Agent
Shared core core Define transport-neutral types, reasoning and fast-tier policy, and HTTP client defaults
Provider adapter core/<provider> Authenticate, convert payloads, select endpoints, and normalize responses
OAuth core/oauth/* Login, token storage, expiry checks, and refresh

Router

router.New takes the text before @ as the provider prefix, drops an optional [tag], looks the prefix up in the factory table, and constructs the Agent. A missing key returns an error. The bracket tag affects naming only, never selection.

Shared policy

core standardizes call parameters while deliberately staying out of the wire payload. ClampReasoning caps Reasoning to each model's range; SupportFast matches Mode against the provider/model whitelist, and only a hit lets the adapter add the native field (Claude's speed, the service_tier used by the OpenAI and Grok families). The served tier is always read back from the response, never assumed from the request.

Provider adapters

Each adapter owns system-prompt merging, message and tool conversion, authentication, endpoint selection, and upstream decoding, exposing only core.Output or core.StreamEvent. OpenAI and Copilot switch between the Chat Completions and Responses APIs per model; Codex and Grok OAuth consume SSE internally and return a completed result; Claude and Copilot additionally expose SendStream.

OAuth lifecycle

OAuth packages persist tokens as JSON in the operating-system keychain. Codex and Grok apply a 60-second safety buffer to expiry checks, then exchange a refresh grant and write the replacement back; Copilot uses the GitHub device flow and refreshes through a session token.

Request flow

  1. The caller builds router.Config with a Name such as openai@gpt-5.4.
  2. router.New selects a factory and constructs an Agent.
  3. The caller invokes Send or SendStream with messages, optional tools, a Reasoning level, and a Mode.
  4. The adapter clamps the reasoning level, checks fast-tier eligibility, shapes the provider payload, and sends the request.
  5. The response is converted to core.Output (or a sequence of StreamEvent values) and returned with the HTTP status code and error.
  6. Usage is unified into Input, Output, CacheCreate, and CacheRead; the served tier lands in Output.ServiceTier.

Further reading

The module-level expansion — per-module diagrams, sequence diagrams, and the OAuth state machine — lives in the repository's doc/architecture.md.

中文