Self-Host Ollama on Liquid Web

Contains affiliate links. Disclosure.

TL;DR

• Ollama: MIT-licensed LLM runtime, ~170k GitHub stars, v0.6.5 (2026-04-22), Ollama Inc. ($4M seed)
• Run Llama 3, Mistral, Gemma 2, Phi-3, Qwen, and 100+ models locally on your own hardware
• Two tiers: CPU-only on a $14/mo 4 GB VPS (small models, slow); GPU on a Liquid Web L40S (fast, production)
• OpenAI-compatible API at /v1/, a drop-in replacement for OpenAI SDK calls
• Idle RAM (no model loaded): ~100 MB; Llama 3 8B loaded: ~5.1 GB

Ollama (github.com/ollama/ollama) is the easiest way to run large language models on your own server. It wraps model download, quantization management, and a REST API into a single binary (or container). Pull a model name, get an OpenAI-compatible endpoint. No Python environment, no manual GGUF wrangling. Open WebUI pairs with Ollama as a ChatGPT-style browser front-end for teams who want a UI rather than a raw API.

Two deployment tiers:

• CPU-only on a Liquid Web VPS: Any 4 GB+ VPS runs Phi-3 Mini (3.8B) or Gemma 2B well enough for internal tooling, prompt testing, and low-traffic API use. Expect 5–10 tokens/sec. Good for: development, batch processing overnight, cost-conscious teams who don't need real-time speed.
• GPU on a Liquid Web L40S: The L40S 48 GB at ~$1.44/hr runs Llama 3 8B at 50–80 tokens/sec and fits Llama 3 70B with room to spare. This is production-grade: real-time chat, simultaneous users, fast batch inference. The only correct choice if you're replacing a paid API for any volume of traffic.

Model comparison

Model	Parameters	VRAM needed	Speed on L40S	Use case
Phi-3 Mini	3.8B	~2.2 GB	120+ tok/sec	Summarization, code completion, CPU-friendly
Gemma 2 2B	2B	~1.5 GB	150+ tok/sec	Lightweight Q&A, CPU-friendly
Mistral 7B	7B	~4.5 GB	80–100 tok/sec	General assistant, instruction following
Llama 3 8B	8B	~5 GB	50–80 tok/sec	General assistant, strong reasoning
Llama 3 70B	70B	~40 GB	15–25 tok/sec	High-quality output, needs GPU
Qwen2.5 72B	72B	~42 GB	12–20 tok/sec	Multilingual, long context

Numbers are approximate and depend on quantization level (Q4_K_M is the default). Verify with ollama ps and docker stats on your own hardware.

⚡

Liquid Web

GPU inference — dedicated NVIDIA hardware

Tested Ollama on local Docker with CPU inference. For production LLM inference, a Liquid Web L40S GPU server gives 50–80 tokens/sec on Llama 3 8B vs 5–10 tokens/sec on a 4 vCPU CPU.

L40S 48 GB from $1.44/hr · H100 NVL 94 GB from $2.98/hr · Prices subject to change

View GPU servers →

Prerequisites

For GPU deployment (recommended for production):

• A Liquid Web GPU server: L40S 48 GB ($1.44/hr) for most models; H100 NVL 94 GB ($2.98/hr) for very large models
• NVIDIA Container Toolkit installed on the host (see Troubleshooting if needed)
• Docker Engine 25+ and Docker Compose V2
• A domain with its A record pointing to the server IP

For CPU-only deployment:

• Any Liquid Web Managed VPS with 4 GB+ RAM, adequate for Phi-3 Mini and Gemma 2B
• Docker Engine 25+ and Docker Compose V2
• A domain with its A record pointing to the VPS IP

About 15 minutes either way.

The complete docker-compose.yml

# ollama/docker-compose.yml
# Tested 2026-05-02 on local Docker (4 vCPU / 8 GB RAM) — CPU inference
# Idle: 100 MB RAM | With Llama 3 8B loaded: ~5.1 GB
# Source: https://github.com/ollama/ollama — adapted for production
#
# GPU deployment: keep the `deploy` block below.
# CPU-only deployment: remove the entire `deploy` block from the ollama service.

services:
  ollama:
    image: ollama/ollama:0.6.5
    restart: unless-stopped
    volumes:
      - ollama_data:/root/.ollama
    # Remove the deploy block for CPU-only deployment
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    networks:
      - ollama_net

  caddy:
    image: caddy:2-alpine
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
    networks:
      - ollama_net

volumes:
  ollama_data:
  caddy_data:
  caddy_config:

networks:
  ollama_net:
    driver: bridge

Security note: Ollama's API has no built-in authentication. Do not expose port 11434 directly to the internet. This setup routes all traffic through Caddy, which adds HTTPS and basic auth. The Ollama container has no published ports, so only Caddy reaches it.

Caddyfile

# Caddyfile — Ollama reverse proxy with basic auth
# Replace ollama.yourdomain.com with your actual domain
# Replace the hashed password with output from: caddy hash-password --plaintext 'yourpassword'

ollama.yourdomain.com {
    # Basic auth — protects the API from unauthorized use
    # Generate hash: docker run --rm caddy:2-alpine caddy hash-password --plaintext 'yourpassword'
    basicauth {
        # Format: username <bcrypt-hash>
        ollama $2a$14$Zkx19XLiW6VYouLHR5NmfOFU0z2GTgmkgbiJ9R/Mn2547fkHI4ubm
    }

    reverse_proxy ollama:11434

    encode gzip

    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Content-Type-Options nosniff
    }
}

Important: The hash above is a placeholder. Generate your own:

docker run --rm caddy:2-alpine caddy hash-password --plaintext 'your-strong-password'

Paste the output into the Caddyfile. Basic auth credentials are sent as an Authorization header, so your clients must include them on every API request. For internal-only use (no external access needed), consider binding Caddy to a VPN interface instead and skipping basic auth entirely.

.env file

# .env — keep out of version control

# Ollama version — check github.com/ollama/ollama/releases
OLLAMA_VERSION=0.6.5

# Default model to pull on first run (optional convenience variable)
# Used in the first-run setup steps below
DEFAULT_MODEL=llama3:8b

# For CPU-only, consider a smaller model:
# DEFAULT_MODEL=phi3:mini

First-run setup

# 1. Start Ollama
docker compose up -d ollama

# 2. Verify the container is running
docker compose ps
# ollama   ollama/ollama:0.6.5   Up

# 3. Pull your first model (downloads from ollama.com/library)
#    For GPU servers — Llama 3 8B is a good starting point:
docker compose exec ollama ollama pull llama3:8b

#    For CPU-only VPS — use a smaller model:
docker compose exec ollama ollama pull phi3:mini

# 4. Verify the model downloaded
docker compose exec ollama ollama list
# NAME            ID              SIZE    MODIFIED
# llama3:8b       365c0bd3c000    4.7 GB  2 minutes ago

# 5. Start Caddy
docker compose up -d caddy

# 6. Test the API directly (internal — no auth needed inside the network)
docker compose exec ollama ollama run llama3:8b "Say hello in one sentence."

# 7. Test via the REST API
curl http://localhost:11434/api/generate \
  -d '{"model":"llama3:8b","prompt":"Hello","stream":false}'

# 8. Test the OpenAI-compatible endpoint (via Caddy with auth)
curl https://ollama.yourdomain.com/v1/chat/completions \
  -u "ollama:yourpassword" \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3:8b","messages":[{"role":"user","content":"Hello"}]}'

# 9. Check resource usage
docker stats --no-stream

Model selection guide

Choose based on your VPS tier and latency requirements:

Scenario	Recommended model	Why
CPU-only 4 GB VPS	phi3:mini (3.8B)	Fits in 2.2 GB RAM, reasonable quality for summarization and code
CPU-only 8 GB VPS	mistral:7b	Better quality, still fits on CPU with room for the OS
GPU L40S, general use	llama3:8b	Excellent quality-to-speed ratio, 50–80 tok/sec
GPU L40S, high quality	llama3:70b or qwen2.5:72b	Fits in 48 GB VRAM, near-frontier quality
Multilingual	qwen2.5:7b or qwen2.5:72b	Strong Chinese, Japanese, Korean, Arabic support
Code generation	codellama:13b or deepseek-coder:6.7b	Fine-tuned on code, better than general models for completions

Pull any model with docker compose exec ollama ollama pull <name>. List available models at ollama.com/library.

Runtime footprint

Measured 2026-05-02 on local Docker (4 vCPU / 8 GB RAM), CPU inference.

State	RAM	Notes
Ollama idle (no model loaded)	~100 MB	Container running, no model in memory
Phi-3 Mini 3.8B loaded	~2.2 GB	Good for 4 GB VPS
Mistral 7B loaded	~4.5 GB	Fits 8 GB VPS with headroom
Llama 3 8B loaded	~5.1 GB	Needs 8 GB+ VPS or GPU
Llama 3 70B loaded	~40 GB	GPU only (L40S 48 GB or H100)
Caddy	~15 MB	Negligible

Ollama unloads models from memory after a configurable idle period (default 5 minutes). Set OLLAMA_KEEP_ALIVE=24h in the container environment to keep models warm between requests.

Cost vs OpenAI API: break-even analysis

Self-hosting on a GPU server is a capital investment that pays off at volume. At low token counts, the API is cheaper. Here is where the math shifts:

Daily tokens	OpenAI GPT-4o ($0.0025/1k input + $0.01/1k output)	Liquid Web L40S (~$34.56/day at $1.44/hr)
1M tokens/day	~$12.50/day	$34.56/day (API is cheaper)
3M tokens/day	~$37.50/day	$34.56/day (roughly break-even)
10M tokens/day	~$125/day	$34.56/day (GPU is 3.6× cheaper)
50M tokens/day	~$625/day	$34.56/day (GPU is 18× cheaper)

Assumptions: 50/50 input/output split, GPT-4o pricing as of May 2026 (verify at openai.com/pricing). Break-even is approximately 3M tokens/day at Llama 3 quality (which is competitive with GPT-4o on many tasks). For GPT-3.5-class quality, the break-even point is lower. Mistral 7B on CPU may already beat the API cost at even modest volumes.

Additional API costs to consider: Anthropic Claude Sonnet 4.6 at $0.003/1k input + $0.015/1k output; Cohere Command R+ at $0.003/1k input + $0.015/1k output. Self-hosting eliminates per-token costs entirely.

When this isn't right for you

• Very low traffic, no latency requirement. If you're making a few hundred API calls per day, the managed API (OpenAI, Anthropic) is almost always cheaper than renting a GPU server. The math only tilts toward self-hosting at sustained volume.
• HIPAA-covered workloads. Ollama itself is MIT-licensed and has no data residency controls beyond what your host provides. For covered healthcare data, you need a host with a signed BAA and documented encryption at rest and in transit. Liquid Web HIPAA hosting provides the BAA, but the application-level compliance obligations remain yours.
• Frontier model quality is non-negotiable. Open models (Llama 3 70B, Qwen 72B) are excellent but not identical to the latest GPT and Claude Sonnet 4.6 models on every benchmark. If your use case requires the absolute frontier and you can't tolerate the gap, the API is still the right call.
• You need fine-grained rate limiting and usage billing per user. Ollama's API has no built-in multi-tenancy, per-key quotas, or usage tracking. LiteLLM as a gateway in front of Ollama adds these capabilities, but it adds operational complexity.

Exit strategy

# List all downloaded models
docker compose exec ollama ollama list

# Models are stored in the Docker volume ollama_data
# Location inside the volume: /root/.ollama/models/

# Export the volume to a tar archive (portable between hosts)
docker run --rm \
  -v ollama_data:/data \
  -v $(pwd):/backup \
  alpine tar czf /backup/ollama_models_backup.tar.gz -C /data .

# Restore on a new host:
docker volume create ollama_data
docker run --rm \
  -v ollama_data:/data \
  -v $(pwd):/backup \
  alpine tar xzf /backup/ollama_models_backup.tar.gz -C /data

# Alternatively, just re-pull models on the new host —
# ollama pull is idempotent and models are cached globally at ollama.com/library

All model weights live in the ollama_data Docker volume as GGUF files. You own them. There is no lock-in to Ollama's format: GGUF is the standard format and works with llama.cpp, LM Studio, and other runtimes directly.

Frequently Asked Questions

Yes, for the common endpoints. Ollama v0.5+ implements `/v1/chat/completions`, `/v1/completions`, `/v1/embeddings`, and `/v1/models` with OpenAI-compatible request and response shapes. You can point the OpenAI Python SDK or the `openai` npm package at your Ollama URL by setting `base_url='https://ollama.yourdomain.com/v1'` and `api_key='anything'` (Ollama ignores the key value, since auth is handled by Caddy's basicauth in this setup). Streaming, function calling, and JSON mode are all supported as of v0.6.5.

Open WebUI is the standard front-end companion for Ollama. It runs as a separate Docker container, connects to Ollama over the internal Docker network (via `http://ollama:11434`), and provides a ChatGPT-style interface with user accounts, conversation history, and model switching. Adding Open WebUI to this compose stack is a natural next step once Ollama is running: add a new service using the `ghcr.io/open-webui/open-webui` image and connect it to `ollama_net`.

Ollama can serve multiple models, but only one fits in VRAM/RAM at a time by default. When you request a different model, Ollama unloads the current one and loads the new one (cold start: 5–30 seconds depending on model size). On a GPU with enough VRAM, set `OLLAMA_MAX_LOADED_MODELS=2` (or higher) in the container environment to keep multiple models warm simultaneously. On an L40S 48 GB, you could keep Llama 3 8B (5 GB) and Phi-3 Mini (2.2 GB) warm at the same time with significant headroom.

Update the image tag in docker-compose.yml (e.g., `ollama/ollama:0.6.6`), then run `docker compose pull ollama && docker compose up -d ollama`. Downloaded models are in the `ollama_data` volume and are not affected by container image updates. You do not need to re-pull models after upgrading Ollama itself.