title	Installation Guide
parent	Getting Started
nav_order	1

Installation Guide

This guide covers all methods to install and run the LLM Proxy.

Prerequisites

Management Token: Generate a secure token for administrative access
```
openssl rand -base64 32
```
OpenAI API Key (optional): Required only if you want to proxy OpenAI requests

Quick Start

The fastest way to get started is with Docker:

docker pull ghcr.io/sofatutor/llm-proxy:latest
mkdir -p ./llm-proxy/data
docker run -d \
  --name llm-proxy \
  -p 8080:8080 \
  -v ./llm-proxy/data:/app/data \
  -e MANAGEMENT_TOKEN=your-secure-management-token \
  ghcr.io/sofatutor/llm-proxy:latest

Verify the installation:

curl http://localhost:8080/health
# Expected: {"status":"ok"}

Installation Methods

Docker (Recommended)

Docker provides the simplest deployment with automatic configuration.

Single Container

Pull and run the official image:

# Pull the latest image
docker pull ghcr.io/sofatutor/llm-proxy:latest

# Create data directory for persistence
mkdir -p ./llm-proxy/data

# Run the container
docker run -d \
  --name llm-proxy \
  -p 8080:8080 \
  -v ./llm-proxy/data:/app/data \
  -e MANAGEMENT_TOKEN=your-secure-management-token \
  ghcr.io/sofatutor/llm-proxy:latest

With Redis Caching

For production deployments, enable Redis-backed caching for better performance:

# Start Redis
docker run -d --name redis -p 6379:6379 redis:alpine

# Start proxy with caching enabled
docker run -d \
  --name llm-proxy \
  -p 8080:8080 \
  -v ./llm-proxy/data:/app/data \
  -e MANAGEMENT_TOKEN=your-secure-management-token \
  -e HTTP_CACHE_ENABLED=true \
  -e HTTP_CACHE_BACKEND=redis \
  -e REDIS_ADDR=redis:6379 \
  --link redis \
  ghcr.io/sofatutor/llm-proxy:latest

Docker Image Tags

Tag	Description
`latest`	Latest stable release
`vX.Y.Z`	Specific version (e.g., `v1.0.0`)
`main`	Latest build from main branch (unstable)

Docker Compose

Docker Compose is recommended for multi-service deployments.

Basic Setup (SQLite)

Create a docker-compose.yml:

services:
  llm-proxy:
    image: ghcr.io/sofatutor/llm-proxy:latest
    container_name: llm-proxy
    ports:
      - "8080:8080"
    volumes:
      - ./data:/app/data
      - ./config:/app/config
    environment:
      - MANAGEMENT_TOKEN=${MANAGEMENT_TOKEN}
      - LOG_LEVEL=info
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "wget", "--no-verbose", "--spider", "http://localhost:8080/health"]
      interval: 30s
      timeout: 5s
      retries: 3

volumes:
  data:

Start the services:

export MANAGEMENT_TOKEN=$(openssl rand -base64 32)
docker compose up -d

With Redis and Event Dispatcher

For production with caching and event logging:

services:
  llm-proxy:
    image: ghcr.io/sofatutor/llm-proxy:latest
    container_name: llm-proxy
    ports:
      - "8080:8080"
    volumes:
      - ./data:/app/data
      - ./logs:/app/logs
      - ./config:/app/config
    environment:
      - MANAGEMENT_TOKEN=${MANAGEMENT_TOKEN}
      - LOG_LEVEL=info
      - HTTP_CACHE_ENABLED=true
      - HTTP_CACHE_BACKEND=redis
      - LLM_PROXY_EVENT_BUS=redis-streams
      - REDIS_ADDR=redis:6379
      - REDIS_DB=0
      - REDIS_STREAM_KEY=llm-proxy-events
      - REDIS_CONSUMER_GROUP=llm-proxy-dispatchers
    depends_on:
      redis:
        condition: service_started
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "wget", "--no-verbose", "--spider", "http://localhost:8080/health"]
      interval: 30s
      timeout: 5s
      retries: 3

  redis:
    image: redis:8
    container_name: llm-proxy-redis
    ports:
      - "6379:6379"
    restart: unless-stopped

  logger:
    image: ghcr.io/sofatutor/llm-proxy:latest
    container_name: llm-proxy-logger
    command: ["dispatcher", "--service", "file", "--endpoint", "/app/logs/events.jsonl"]
    volumes:
      - ./logs:/app/logs
    environment:
      - LLM_PROXY_EVENT_BUS=redis-streams
      - REDIS_ADDR=redis:6379
      - REDIS_STREAM_KEY=llm-proxy-events
      - REDIS_CONSUMER_GROUP=llm-proxy-dispatchers
    depends_on:
      - redis

volumes:
  data:
  logs:

With PostgreSQL

For production deployments requiring a robust database:

services:
  postgres:
    image: postgres:15
    container_name: llm-proxy-postgres
    environment:
      POSTGRES_USER: llmproxy
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: llmproxy
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U llmproxy -d llmproxy"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: unless-stopped

  llm-proxy:
    image: ghcr.io/sofatutor/llm-proxy:latest
    container_name: llm-proxy
    ports:
      - "8080:8080"
    volumes:
      - ./config:/app/config
    environment:
      - MANAGEMENT_TOKEN=${MANAGEMENT_TOKEN}
      - DB_DRIVER=postgres
      - DATABASE_URL=postgres://llmproxy:${POSTGRES_PASSWORD}@postgres:5432/llmproxy?sslmode=disable
      - LOG_LEVEL=info
    depends_on:
      postgres:
        condition: service_healthy
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "wget", "--no-verbose", "--spider", "http://localhost:8080/health"]
      interval: 30s
      timeout: 5s
      retries: 3

volumes:
  postgres_data:

Start with:

export MANAGEMENT_TOKEN=$(openssl rand -base64 32)
export POSTGRES_PASSWORD=$(openssl rand -base64 16)
docker compose up -d

See Docker Compose PostgreSQL Setup for detailed PostgreSQL configuration.

Build from Source

Building from source requires Go 1.23 or later.

Requirements

Go 1.23+
Make
Git

Build Steps

# Clone the repository
git clone https://github.com/sofatutor/llm-proxy.git
cd llm-proxy

# Install dependencies
make dev-setup

# Build the binary (SQLite only)
make build

# Or build with database support
# With PostgreSQL support
go build -tags postgres -o bin/llm-proxy ./cmd/proxy

# With MySQL support
go build -tags mysql -o bin/llm-proxy ./cmd/proxy

# With both PostgreSQL and MySQL support
go build -tags "postgres,mysql" -o bin/llm-proxy ./cmd/proxy

# The binary is created at ./bin/llm-proxy

Running

# Set required environment variables
export MANAGEMENT_TOKEN=$(openssl rand -base64 32)

# Start the server with SQLite (default)
./bin/llm-proxy server

# Or start with PostgreSQL (requires -tags postgres build)
DB_DRIVER=postgres DATABASE_URL=postgres://... ./bin/llm-proxy server

# Or start with MySQL (requires -tags mysql build)
DB_DRIVER=mysql DATABASE_URL=user:pass@tcp(host:port)/db?parseTime=true ./bin/llm-proxy server

# Or start with custom options
./bin/llm-proxy server --addr :9000 --db ./custom/path/db.sqlite

Available Make Commands

Command	Description
`make build`	Build the binary
`make test`	Run all tests
`make lint`	Run linters
`make fmt`	Format code
`make docker-build`	Build Docker image locally
`make dev-setup`	Install development tools

Pre-built Binaries

Pre-built binaries are available from GitHub Releases.

Download and Install

# Download the latest release (adjust version and platform)
VERSION=v1.0.0
PLATFORM=linux-amd64

curl -L -o llm-proxy.tar.gz \
  "https://github.com/sofatutor/llm-proxy/releases/download/${VERSION}/llm-proxy-${PLATFORM}.tar.gz"

# Extract
tar -xzf llm-proxy.tar.gz

# Move to a location in PATH
sudo mv llm-proxy /usr/local/bin/

# Verify
llm-proxy --version

Available Platforms

Platform	Architecture	File
Linux	amd64	`llm-proxy-linux-amd64.tar.gz`
Linux	arm64	`llm-proxy-linux-arm64.tar.gz`
macOS	amd64	`llm-proxy-darwin-amd64.tar.gz`
macOS	arm64 (M1/M2)	`llm-proxy-darwin-arm64.tar.gz`
Windows	amd64	`llm-proxy-windows-amd64.zip`

Verification Steps

After installation, verify your setup:

1. Health Check

curl http://localhost:8080/health
# Expected: {"status":"ok"}

2. Create a Test Project

curl -X POST http://localhost:8080/manage/projects \
  -H "Authorization: Bearer $MANAGEMENT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "Test Project", "openai_api_key": "sk-test-key"}'

3. Generate a Token

PROJECT_ID=<project-id-from-above>
curl -X POST http://localhost:8080/manage/tokens \
  -H "Authorization: Bearer $MANAGEMENT_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"project_id\": \"$PROJECT_ID\", \"duration_hours\": 24}"

4. Access the Admin UI

Open http://localhost:8080/admin/ in your browser.

Platform-Specific Notes

Linux

File Permissions: Ensure the data directory is writable
```
mkdir -p ./data && chmod 755 ./data
```
Systemd Service: See Production Deployment for systemd configuration
SELinux: If using SELinux, configure appropriate contexts for mounted volumes

macOS

Docker Desktop: Ensure Docker Desktop is running before using Docker commands
Apple Silicon (M1/M2): Use darwin-arm64 binaries or Docker images (multi-arch supported)
Network Access: Allow incoming connections when prompted by the firewall

Windows

Docker Desktop: Use WSL 2 backend for best performance

PowerShell: Use PowerShell for environment variable commands

$env:MANAGEMENT_TOKEN = "your-token"
.\llm-proxy.exe server

Paths: Use forward slashes in Docker volume paths
```
docker run -v "${PWD}/data:/app/data" ...
```

Troubleshooting Installation

Docker Issues

Container won't start:

# Check container logs
docker logs llm-proxy

# Common causes:
# - Port 8080 already in use
# - Missing MANAGEMENT_TOKEN environment variable
# - Volume mount permission issues

Permission denied on volumes:

# Fix ownership (Linux)
sudo chown -R $(id -u):$(id -g) ./data

Build Issues

Go version mismatch:

# Check Go version
go version
# Required: go1.23 or later

# Update Go from https://go.dev/dl/

Missing dependencies:

# Download dependencies
go mod download
go mod tidy

Connection Issues

Cannot connect to proxy:

# Check if proxy is running
curl -v http://localhost:8080/health

# Check if port is in use
lsof -i :8080  # macOS/Linux
netstat -an | findstr :8080  # Windows

Next Steps

After installation:

Configuration Guide - Configure environment variables and settings
Admin UI Quickstart - Set up projects and tokens via the web interface
Token Management - Understand token lifecycle and management
Security Best Practices - Secure your deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Installation Guide

Prerequisites

Quick Start

Installation Methods

Docker (Recommended)

Single Container

With Redis Caching

Docker Image Tags

Docker Compose

Basic Setup (SQLite)

With Redis and Event Dispatcher

With PostgreSQL

Build from Source

Requirements

Build Steps

Running

Available Make Commands

Pre-built Binaries

Download and Install

Available Platforms

Verification Steps

1. Health Check

2. Create a Test Project

3. Generate a Token

4. Access the Admin UI

Platform-Specific Notes

Linux

macOS

Windows

Troubleshooting Installation

Docker Issues

Build Issues

Connection Issues

Next Steps

Related Documentation

FilesExpand file tree

installation.md

Latest commit

History

installation.md

File metadata and controls

Installation Guide

Prerequisites

Quick Start

Installation Methods

Docker (Recommended)

Single Container

With Redis Caching

Docker Image Tags

Docker Compose

Basic Setup (SQLite)

With Redis and Event Dispatcher

With PostgreSQL

Build from Source

Requirements

Build Steps

Running

Available Make Commands

Pre-built Binaries

Download and Install

Available Platforms

Verification Steps

1. Health Check

2. Create a Test Project

3. Generate a Token

4. Access the Admin UI

Platform-Specific Notes

Linux

macOS

Windows

Troubleshooting Installation

Docker Issues

Build Issues

Connection Issues

Next Steps

Related Documentation