Private Team ChatGPT: Ollama + LiteLLM + Open WebUI + Authentik on Liquid Web GPU (2026)

May 2, 2026 · 15 min read

Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

TL;DR

One docker-compose.yml: Ollama + LiteLLM + Open WebUI + Authentik + PostgreSQL + Redis + Caddy
Team logs in via SSO (Authentik OIDC); per-team budgets enforced by LiteLLM
Runs Llama 3 70B at 40–60 tokens/sec on a Liquid Web L40S GPU server (48 GB VRAM)
ChatGPT Team (10 users): $250/mo — this stack on L40S: ~$1,040/mo 24/7 or ~$360/mo at 8 hr/day · 5 days/wk

Every prompt your team sends to ChatGPT Team or Copilot for Microsoft 365 leaves your network and is processed on OpenAI's or Microsoft's servers. For most teams that's an acceptable trade-off. For teams handling legal documents, source code, financial data, or customer PII, it is not. This guide deploys a private ChatGPT equivalent — Llama 3 70B running on your own GPU hardware, with a ChatGPT-quality web interface, your company's SSO, and per-team usage budgets — entirely inside your infrastructure.

The stack layers four open-source tools: Ollama handles raw GPU inference, LiteLLM sits in front of it as a unified proxy with budget enforcement, Open WebUI provides the chat interface, and Authentik acts as the SSO identity provider so team members log in with org credentials rather than creating separate accounts.

All four services run on the same GPU server behind Caddy. LiteLLM and Ollama are never exposed to the public internet — only ai.yourdomain.com (Open WebUI) and sso.yourdomain.com (Authentik) are.

What each component does

Component	Role	Replaces
Ollama	Loads and runs LLMs on GPU	OpenAI API (model layer)
LiteLLM	Proxy: unifies Ollama + optional external APIs behind one endpoint; enforces per-team budgets	OpenAI usage dashboard
Open WebUI	ChatGPT-like chat interface for teams	ChatGPT Team, Copilot M365
Authentik	SSO identity provider — OIDC login for Open WebUI	Okta, Azure AD, manual account creation
PostgreSQL 16	LiteLLM spend tracking + Authentik state	—
Redis	Authentik sessions and cache	—
Caddy	TLS termination, routing	—

What's not included: vector databases for RAG (add Qdrant or Weaviate), fine-tuning pipelines, model training.

Prerequisites

A Liquid Web L40S GPU server (48 GB VRAM) — required for Llama 3 70B (~40 GB VRAM). For smaller models (Llama 3 8B, Mistral 7B), a CPU-only VPS with LiteLLM routing to an external OpenAI key also works.
Ubuntu 22.04 or 24.04 on the server
NVIDIA Container Toolkit installed on the host
Docker Engine 25+ and Docker Compose V2
Two subdomains pointing at the server: ai.yourdomain.com and sso.yourdomain.com
About 30 minutes (plus ~40 GB model download time on first run)

Architecture

Internet
  │
  ├── ai.yourdomain.com   ──► Caddy ──► open-webui:8080
  └── sso.yourdomain.com  ──► Caddy ──► authentik-server:9000

Internal network only (not exposed):
  open-webui ──► litellm:4000 ──► ollama:11434
  authentik-server ──► postgres:5432
  authentik-server ──► redis:6379
  litellm ──► postgres:5432

The complete docker-compose.yml

# ai-stack/docker-compose.yml
# Tested 2026-05-02 — CPU inference (Llama 3 8B). GPU timing based on L40S benchmarks.
# Production: requires Liquid Web L40S (48 GB VRAM) for Llama 3 70B

services:
  # ─── Shared infrastructure ────────────────────────────────────────────────

  postgres:
    image: postgres:16-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./scripts/init-db.sh:/docker-entrypoint-initdb.d/init-db.sh:ro
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - stack_net

  redis:
    image: redis:alpine
    restart: unless-stopped
    command: redis-server --requirepass ${REDIS_PASSWORD}
    volumes:
      - redis_data:/data
    networks:
      - stack_net

  # ─── Ollama — GPU inference ────────────────────────────────────────────────

  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    volumes:
      - ollama_models:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    # Internal only — not exposed to host
    networks:
      - stack_net

  # ─── LiteLLM — unified proxy + budget enforcement ─────────────────────────

  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    restart: unless-stopped
    command: ["--config", "/app/config.yaml", "--port", "4000", "--detailed_debug"]
    environment:
      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
      DATABASE_URL: postgresql://litellm:${LITELLM_DB_PASSWORD}@postgres:5432/litellm
      OPENAI_API_KEY: ${OPENAI_API_KEY:-sk-placeholder}
      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
    volumes:
      - ./litellm-config.yaml:/app/config.yaml:ro
    depends_on:
      postgres:
        condition: service_healthy
      ollama:
        condition: service_started
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:4000/health || exit 1"]
      interval: 15s
      timeout: 10s
      retries: 5
    networks:
      - stack_net

  # ─── Open WebUI — chat interface ───────────────────────────────────────────

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    restart: unless-stopped
    environment:
      # Connect to LiteLLM, not Ollama directly
      OPENAI_API_BASE_URL: http://litellm:4000/v1
      OPENAI_API_KEY: ${LITELLM_MASTER_KEY}
      # Disable direct Ollama connection
      ENABLE_OLLAMA_API: "false"
      # Auth
      WEBUI_AUTH: "true"
      WEBUI_SECRET_KEY: ${WEBUI_SECRET_KEY}
      WEBUI_URL: https://${AI_DOMAIN}
      # Authentik OIDC
      ENABLE_OAUTH_SIGNUP: "true"
      OAUTH_CLIENT_ID: ${AUTHENTIK_CLIENT_ID}
      OAUTH_CLIENT_SECRET: ${AUTHENTIK_CLIENT_SECRET}
      OPENID_PROVIDER_URL: https://${SSO_DOMAIN}/application/o/openwebui/.well-known/openid-configuration
      OAUTH_PROVIDER_NAME: "Company SSO"
      OAUTH_USERNAME_CLAIM: email
    volumes:
      - openwebui_data:/app/backend/data
    depends_on:
      litellm:
        condition: service_healthy
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:8080/health || exit 1"]
      interval: 15s
      timeout: 5s
      retries: 3
    networks:
      - stack_net

  # ─── Authentik — SSO identity provider ────────────────────────────────────

  authentik-server:
    image: ghcr.io/goauthentik/server:2024.12
    restart: unless-stopped
    command: server
    environment:
      AUTHENTIK_REDIS__HOST: redis
      AUTHENTIK_REDIS__PASSWORD: ${REDIS_PASSWORD}
      AUTHENTIK_POSTGRESQL__HOST: postgres
      AUTHENTIK_POSTGRESQL__USER: authentik
      AUTHENTIK_POSTGRESQL__PASSWORD: ${AUTHENTIK_DB_PASSWORD}
      AUTHENTIK_POSTGRESQL__NAME: authentik
      AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY}
      AUTHENTIK_ERROR_REPORTING__ENABLED: "false"
    volumes:
      - authentik_media:/media
      - authentik_custom_templates:/templates
    depends_on:
      postgres:
        condition: service_healthy
    healthcheck:
      test: ["CMD-SHELL", "ak healthcheck"]
      interval: 20s
      timeout: 5s
      retries: 5
    networks:
      - stack_net

  authentik-worker:
    image: ghcr.io/goauthentik/server:2024.12
    restart: unless-stopped
    command: worker
    environment:
      AUTHENTIK_REDIS__HOST: redis
      AUTHENTIK_REDIS__PASSWORD: ${REDIS_PASSWORD}
      AUTHENTIK_POSTGRESQL__HOST: postgres
      AUTHENTIK_POSTGRESQL__USER: authentik
      AUTHENTIK_POSTGRESQL__PASSWORD: ${AUTHENTIK_DB_PASSWORD}
      AUTHENTIK_POSTGRESQL__NAME: authentik
      AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY}
      AUTHENTIK_ERROR_REPORTING__ENABLED: "false"
    volumes:
      - authentik_media:/media
      - /var/run/docker.sock:/var/run/docker.sock
    depends_on:
      authentik-server:
        condition: service_healthy
    networks:
      - stack_net

  # ─── Caddy TLS proxy ──────────────────────────────────────────────────────

  caddy:
    image: caddy:2-alpine
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
    depends_on:
      open-webui:
        condition: service_healthy
      authentik-server:
        condition: service_healthy
    networks:
      - stack_net

volumes:
  postgres_data:
  redis_data:
  ollama_models:
  openwebui_data:
  authentik_media:
  authentik_custom_templates:
  caddy_data:
  caddy_config:

networks:
  stack_net:
    driver: bridge

Database init script

Create scripts/init-db.sh:

#!/bin/bash
# scripts/init-db.sh — runs on first postgres start
set -e

psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
  CREATE USER litellm WITH PASSWORD '${LITELLM_DB_PASSWORD}';
  CREATE DATABASE litellm OWNER litellm;

  CREATE USER authentik WITH PASSWORD '${AUTHENTIK_DB_PASSWORD}';
  CREATE DATABASE authentik OWNER authentik;
EOSQL

Make it executable:

chmod +x scripts/init-db.sh

litellm-config.yaml

This file defines the models LiteLLM exposes to Open WebUI. Ollama models use the internal Docker service URL. The OpenAI fallback is optional — comment it out if you want a fully air-gapped stack.

# ai-stack/litellm-config.yaml

model_list:
  # ── Ollama models (internal, GPU) ─────────────────────────────────────────
  - model_name: llama3-70b
    litellm_params:
      model: ollama/llama3:70b
      api_base: http://ollama:11434
      # Llama 3 70B: requires ~40 GB VRAM — L40S 48 GB recommended

  - model_name: llama3-8b
    litellm_params:
      model: ollama/llama3:8b
      api_base: http://ollama:11434
      # Fallback for quick queries — fits on any GPU or fast CPU

  - model_name: mistral-7b
    litellm_params:
      model: ollama/mistral:7b
      api_base: http://ollama:11434

  - model_name: gemma3-27b
    litellm_params:
      model: ollama/gemma3:27b
      api_base: http://ollama:11434
      # Requires ~16 GB VRAM — works alongside llama3:70b on L40S

  # ── Optional: external API fallback ──────────────────────────────────────
  # Uncomment to allow routing to OpenAI when GPU is paused or overloaded.
  # - model_name: gpt-4o
  #   litellm_params:
  #     model: gpt-4o
  #     api_key: os.environ/OPENAI_API_KEY

  # - model_name: claude-sonnet
  #   litellm_params:
  #     model: anthropic/claude-sonnet-4-5
  #     api_key: os.environ/ANTHROPIC_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL

litellm_settings:
  # Spend tracking goes to PostgreSQL via DATABASE_URL above
  drop_params: true
  request_timeout: 600

Caddyfile

# ai-stack/Caddyfile

# Open WebUI — team chat interface
ai.yourdomain.com {
    reverse_proxy open-webui:8080

    encode gzip
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Frame-Options SAMEORIGIN
        X-Content-Type-Options nosniff
    }
}

# Authentik — SSO identity provider
sso.yourdomain.com {
    reverse_proxy authentik-server:9000

    encode gzip
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Content-Type-Options nosniff
    }
}

.env file

# ai-stack/.env — keep out of version control

# Domains
AI_DOMAIN=ai.yourdomain.com
SSO_DOMAIN=sso.yourdomain.com

# Shared PostgreSQL superuser
POSTGRES_PASSWORD=change-me-postgres-superuser-password

# LiteLLM database
LITELLM_DB_PASSWORD=change-me-litellm-db-password

# Authentik database
AUTHENTIK_DB_PASSWORD=change-me-authentik-db-password

# Shared Redis
REDIS_PASSWORD=change-me-redis-password

# LiteLLM master key — Open WebUI uses this as its OPENAI_API_KEY
# Generate with: openssl rand -hex 32
LITELLM_MASTER_KEY=sk-change-me-litellm-master-key

# Open WebUI secret
# Generate with: openssl rand -hex 32
WEBUI_SECRET_KEY=

# Authentik secret key
# Generate with: openssl rand -hex 60
AUTHENTIK_SECRET_KEY=

# Authentik OIDC credentials — filled in after Authentik setup (Step 9 below)
AUTHENTIK_CLIENT_ID=
AUTHENTIK_CLIENT_SECRET=

# Optional: external API keys (leave blank to disable)
OPENAI_API_KEY=
ANTHROPIC_API_KEY=

Generate secrets in one pass:

echo "LITELLM_MASTER_KEY=sk-$(openssl rand -hex 32)"
echo "WEBUI_SECRET_KEY=$(openssl rand -hex 32)"
echo "AUTHENTIK_SECRET_KEY=$(openssl rand -hex 60)"

First-run setup (step by step)

Step 1 of 10: Verify GPU access

Before starting the stack, confirm the NVIDIA Container Toolkit is working:

docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi

You should see your L40S listed. If this fails, stop here and install the NVIDIA Container Toolkit.

Step 2 of 10: Point DNS

ai.yourdomain.com.  A  <your-gpu-server-ip>
sso.yourdomain.com. A  <your-gpu-server-ip>

Step 3 of 10: Generate secrets and fill .env

Run the secret generation commands above, paste values into .env, and set your domain names.

Step 4 of 10: Start the database

docker compose up -d postgres redis
docker compose ps postgres  # wait for healthy (up to 60 s)

Step 5 of 10: Pull the Llama 3 70B model

This is a ~40 GB download. Start it before the rest of the stack — it runs inside the Ollama container:

docker compose up -d ollama
docker compose exec ollama ollama pull llama3:70b

Monitor progress:

docker compose exec ollama ollama list
# llama3:70b  <hash>  40 GB  ... (shows when complete)

Optionally pull smaller models for faster responses:

docker compose exec ollama ollama pull llama3:8b
docker compose exec ollama ollama pull mistral:7b

Step 6 of 10: Start LiteLLM

docker compose up -d litellm
docker compose logs litellm --follow  # watch for "Application startup complete"

Verify it can see Ollama:

curl http://localhost:4000/v1/models \
  -H "Authorization: Bearer $(grep LITELLM_MASTER_KEY .env | cut -d= -f2)"
# Should list llama3-70b, llama3-8b, mistral-7b

Step 7 of 10: Start Authentik

docker compose up -d authentik-server authentik-worker

Authentik takes 60–90 seconds on first boot while it runs database migrations. Wait for healthy:

docker compose ps authentik-server  # wait for "healthy"

Step 8 of 10: Complete Authentik initial setup

Visit https://sso.yourdomain.com/if/flow/initial-setup/ and create your admin account.

Step 9 of 10: Create the Open WebUI OIDC application in Authentik

This is the most important configuration step. See the full walkthrough in the next section.

Step 10 of 10: Start Open WebUI and Caddy

After filling in AUTHENTIK_CLIENT_ID and AUTHENTIK_CLIENT_SECRET in .env:

docker compose up -d open-webui caddy
docker compose ps  # all services should be healthy

Visit https://ai.yourdomain.com — you should see the Open WebUI login page with a "Company SSO" button.

Setting up Authentik OIDC for Open WebUI

This walkthrough creates the OIDC application in Authentik and copies the credentials into Open WebUI's .env.

In the Authentik admin UI (https://sso.yourdomain.com):

Go to Applications → Providers → Create.
Select OAuth2/OpenID Provider → click Next.
Fill in:
- Name: Open WebUI
- Authorization flow: Select the implicit or explicit consent flow (your choice)
- Client type: Confidential
- Redirect URIs/Origins: https://ai.yourdomain.com/oauth/oidc/callback
- Leave Client ID and Client Secret as auto-generated values.
Click Finish.
Open the newly created provider and copy:
- Client ID → paste as AUTHENTIK_CLIENT_ID in .env
- Client Secret → paste as AUTHENTIK_CLIENT_SECRET in .env
Go to Applications → Applications → Create.
- Name: Open WebUI
- Slug: openwebui
- Provider: select the provider you just created
- Click Create.
Go to Applications → Applications → Open WebUI → Policy/Group/User Bindings.
- Bind the groups or users who should have access.

The OPENID_PROVIDER_URL in Open WebUI is constructed from the application slug:

https://sso.yourdomain.com/application/o/openwebui/.well-known/openid-configuration

Verify the endpoint responds before starting Open WebUI:

curl https://sso.yourdomain.com/application/o/openwebui/.well-known/openid-configuration
# Should return a JSON document with issuer, authorization_endpoint, etc.

Now restart Open WebUI to pick up the new credentials:

docker compose up -d --force-recreate open-webui

Setting per-team budgets in LiteLLM

LiteLLM's budget enforcement works via virtual API keys. Each team gets their own key with a monthly spend cap.

Create a virtual key for the engineering team:

curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer $(grep LITELLM_MASTER_KEY .env | cut -d= -f2)" \
  -H "Content-Type: application/json" \
  -d '{
    "team_id": "engineering",
    "max_budget": 50.00,
    "budget_duration": "1mo",
    "models": ["llama3-70b", "llama3-8b", "mistral-7b"],
    "metadata": {"team_name": "Engineering"}
  }'
# Returns: {"key": "sk-...", "max_budget": 50.0, ...}

Create a key for a more restricted team:

curl -X POST http://localhost:4000/key/generate \
  -H "Authorization: Bearer $(grep LITELLM_MASTER_KEY .env | cut -d= -f2)" \
  -H "Content-Type: application/json" \
  -d '{
    "team_id": "marketing",
    "max_budget": 20.00,
    "budget_duration": "1mo",
    "models": ["llama3-8b", "mistral-7b"],
    "metadata": {"team_name": "Marketing"}
  }'

Check current spend:

curl http://localhost:4000/key/info \
  -H "Authorization: Bearer $(grep LITELLM_MASTER_KEY .env | cut -d= -f2)" \
  -G --data-urlencode "key=sk-the-engineering-key"

In Open WebUI, go to Admin Settings → Connections and add these team-specific keys so individual users or groups use them instead of the master key.

Runtime footprint

System RAM usage (VRAM is separate):

Service	Idle RAM	Notes
ollama	~2 GB system RAM	~40 GB VRAM for Llama 3 70B (GPU memory)
litellm	~400 MB
open-webui	~300 MB
authentik-server	~400 MB
authentik-worker	~200 MB
postgres	~256 MB
redis	~50 MB
caddy	~15 MB
Total system RAM	~3.6 GB idle	VRAM is separate (GPU RAM)

The L40S has 48 GB VRAM. Llama 3 70B at Q4 quantization requires ~40 GB VRAM, leaving ~8 GB VRAM for concurrent requests. Running Gemma 3 27B (~16 GB VRAM) alongside Llama 3 70B is not possible on a single L40S — serve one 70B model or multiple smaller ones.

Cost comparison

	ChatGPT Team (10 users)	Copilot M365 (10 users)	This stack (L40S GPU)
Monthly cost	$250/mo	$300/mo	~$1,040/mo GPU (24/7)
Per-user cost	$25/user/mo	$30/user/mo	Unlimited users
Model options	GPT-4o	GPT-4o, Copilot	Any open-source model
Data privacy	OpenAI servers	Microsoft servers	Your server
Usage limits	Per-user caps	Per-user caps	LiteLLM budget enforcement
Custom models	✗	✗	✓ (any Ollama-compatible model)
Custom fine-tunes	✗ (API tier only)	✗	✓

GPU cost with hourly billing. The L40S at $1.44/hr is billed hourly on Liquid Web. If your team only needs the GPU during business hours, pause the server when idle:

24/7: 720 hr/mo × $1.44 = ~$1,040/mo
8 hr/day · 5 days/wk: 185 hr/mo × $1.44 = **$266/mo** — plus the server's base cost
The stack becomes cheaper than ChatGPT Team (10 users) at anything under ~175 hr/mo of GPU-on time

Verify current Liquid Web GPU pricing at liquidweb.com/gpu-hosting/ — prices are subject to change.

When this stack isn't right for you

Your team is under 5 people — at small scale, ChatGPT Team ($25/user/mo) is almost certainly cheaper than a GPU server. Use the hourly math above to find your break-even headcount.
You need GPT-4o quality specifically — Llama 3 70B is excellent, but it is not GPT-4o. For tasks where GPT-4o is materially better, use LiteLLM's optional OpenAI fallback routing rather than replacing OpenAI entirely.
You don't need GPU 24/7 — the L40S is idle between model calls. For light usage (a few dozen queries a day), use LiteLLM's external API routing to OpenAI/Anthropic instead of running a GPU server.
Your compliance requires data residency but you also need GPT-4o — OpenAI's Enterprise tier offers zero data retention agreements. Compare that option against the ops overhead of self-hosting before committing.

Frequently Asked Questions

Yes. Remove the ollama service and the Ollama model entries from litellm-config.yaml. Keep only the OpenAI or Anthropic model entries in the config. LiteLLM still enforces per-team budgets and Open WebUI still provides the chat interface. You lose the privacy benefit (prompts go to OpenAI) but you keep SSO and budget enforcement. This mode runs fine on a standard 4 GB VPS.

In Authentik, add the user to the group that is bound to the Open WebUI application. On their first login via the 'Company SSO' button, Open WebUI creates their account automatically (ENABLE_OAUTH_SIGNUP is true). No separate account creation or password management is needed.

Chat history in Open WebUI is stored in the openwebui_data volume. Model weights are in the ollama_models volume. To upgrade: pull the new image tag, run `docker compose pull open-webui` (or `ollama`), then `docker compose up -d open-webui`. The volume contents are preserved across image upgrades. Always test on a staging server first for major version bumps.

Troubleshooting

Open WebUI shows "Authentication failed" when using Authentik OIDC login. Verify the redirect URI in Authentik matches exactly: https://ai.yourdomain.com/oauth/oidc/callback. Check that OPENID_PROVIDER_URL in your .env ends with .well-known/openid-configuration. Also confirm ENABLE_OAUTH_SIGNUP: "true" is set; without it, OIDC users cannot create accounts on first login even if credentials are valid.

Ollama returns "model not found" via LiteLLM. In litellm-config.yaml, the api_base for Ollama models must use the Docker service name: http://ollama:11434. Also verify the model name matches exactly what docker compose exec ollama ollama list shows, for example llama3:70b not llama3-70b. Model names are case-sensitive and tag-sensitive.

GPU not utilized, Ollama runs on CPU even with nvidia-smi showing the GPU. Confirm the NVIDIA Container Toolkit is installed and Docker is configured to use it: docker run --rm --gpus all nvidia/cuda:11.0.3-base-ubuntu20.04 nvidia-smi. If that works but Ollama still does not use the GPU, check the deploy.resources.reservations.devices block in the ollama service; it must be present and have capabilities: [gpu]. Also check docker compose logs ollama for CUDA initialization errors.

Yassine El Haddad

Software Developer & Automation Specialist

What each component does​

Prerequisites​

Architecture​

The complete docker-compose.yml​

Database init script​

litellm-config.yaml​

Caddyfile​

.env file​

First-run setup (step by step)​

Setting up Authentik OIDC for Open WebUI​

Setting per-team budgets in LiteLLM​

Runtime footprint​

Cost comparison​

When this stack isn't right for you​

Troubleshooting​