Documentation v0.4.0
Home GitHub

Getting Started

Install go-llm-router and send a first request through the unified Agent interface.

Requirements

Install

go get github.com/pardnchiu/go-llm-router

Build from source:

git clone https://github.com/pardnchiu/go-llm-router.git
cd go-llm-router
go build ./...

Send a first request

Send takes five arguments: a context, messages, tool definitions (nil is allowed), a Reasoning level, and a Mode. Every provider shares the same arguments, so switching models does not change the request loop.

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/pardnchiu/go-llm-router/core"
    "github.com/pardnchiu/go-llm-router/core/router"
)

func main() {
    agent, err := router.New(router.Config{
        Name:   "openai@gpt-5.4",
        APIKey: "your-api-key",
    })
    if err != nil {
        log.Fatal(err)
    }

    messages := []core.Message{
        {Role: "user", Content: "Explain Go interfaces in one sentence."},
    }

    out, status, err := agent.Send(
        context.Background(),
        messages,
        nil,
        core.ReasoningMedium,
        core.ModeDefault,
    )
    if err != nil {
        log.Fatalf("request failed (HTTP %d): %v", status, err)
    }
    if len(out.Choices) == 0 {
        log.Fatal("provider returned no choices")
    }

    fmt.Println(out.Choices[0].Message.Content)
}

Name uses the <provider>@<model> format. An unknown provider prefix makes router.New return an error.

Request the fast tier

ModeFast is a capability request, not a guarantee. Check the provider/model pair with core.SupportFast first; unsupported pairs stay on the standard tier silently instead of failing.

mode := core.ModeDefault
if core.SupportFast("openai", "gpt-5.4") {
    mode = core.ModeFast
}

out, status, err := agent.Send(ctx, messages, nil, core.ReasoningHigh, mode)

Stream output

Providers that implement core.StreamAgent deliver text, reasoning, tool-call, usage, completion, and error events through a channel.

streamer, ok := agent.(core.StreamAgent)
if !ok {
    return fmt.Errorf("%s does not support streaming", agent.Name())
}

events, err := streamer.SendStream(ctx, messages, nil, core.ReasoningMedium, core.ModeDefault)
if err != nil {
    return err
}

for event := range events {
    switch event.Type {
    case core.StreamEventText:
        fmt.Print(event.TextDelta)
    case core.StreamEventReasoning:
        fmt.Print(event.ReasoningDelta)
    case core.StreamEventError:
        return event.Err
    }
}

Local test server

make test runs go run ./cmd/test, an OpenAI-compatible HTTP server listening on 8787 by default; override it with PORT. Credentials come from environment variables.

export OPENAI_API_KEY="your-api-key"
make test

curl http://127.0.0.1:8787/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai@gpt-5.4",
    "messages": [{"role": "user", "content": "Hello"}],
    "reasoning": "medium",
    "mode": "default"
  }'

Adding "stream": true switches the response to OpenAI-style chat.completion.chunk SSE records ending with data: [DONE].

Next steps

中文