Ollama + Open WebUI: Build a Private ChatGPT Clone on Your Own Server (2026)

Contains affiliate links. Disclosure.

Ollama runs open-source LLMs locally. Open WebUI provides a ChatGPT-like interface that connects to Ollama (or OpenAI/Anthropic). Together they give you a fully private chat experience — no data leaves your server. This guide covers installation, model choices by hardware, and production setup on CPU or GPU VPS.

What Are Ollama and Open WebUI?

Ollama — a local LLM runtime. You run ollama pull llama3.2 and ollama run llama3.2 to chat in the terminal. Ollama exposes an HTTP API on port 11434 compatible with OpenAI's format.

Open WebUI — a web frontend (chat UI, model switching, RAG, plugins) that talks to Ollama, OpenAI, or Anthropic. It looks and feels like ChatGPT. When pointed at Ollama, all processing stays on your machine.

Hardware Options

Setup	Speed	Best for
CPU only	1–5 tok/s	Testing, light use, small models (3B)
Consumer GPU (RTX 3090, 4090)	30–80 tok/s	Personal use, 7B–70B models
Cloud GPU VPS (A10, L4, A100)	20–100+ tok/s	Production, multi-user, always-on

For CPU-only, use 3B–7B quantized models. For 8 GB VRAM: 7B models. For 16 GB: 7B–13B. For 24 GB+: 70B quantized. Liquid Web GPU offers hourly billing for bursty workloads.

Installation

Ollama (Linux)

curl -fsSL https://ollama.com/install.sh | sh
ollama serve   # Or: systemctl enable ollama && systemctl start ollama

Pull a Model

ollama pull llama3.2        # ~4.7 GB, 3B default
ollama pull llama3.2:7b    # ~4.3 GB, 7B
ollama pull mistral        # ~4.1 GB
ollama pull phi3:mini      # ~2.3 GB, good for low VRAM
ollama pull codestral      # Coding-focused

Test in CLI:

ollama run llama3.2

Open WebUI (Docker)

Ollama on host, Open WebUI in Docker:

docker run -d -p 3000:8080 \
  -e OLLAMA_BASE_URL=http://host.docker.internal:11434 \
  --add-host=host.docker.internal:host-gateway \
  -v open-webui:/app/backend/data \
  ghcr.io/open-webui/open-webui:main

Both in Docker — use docker-compose:

services:
  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - ollama:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]   # Omit for CPU-only

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    restart: unless-stopped
    ports:
      - "3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama

volumes:
  ollama:
  open-webui:

For CPU-only, remove the deploy.resources block from ollama.

Recommended Models by Hardware

VRAM / RAM	Recommended Models	Use Case
4 GB	phi3:mini, tinyllama, llama3.2:3b	Light chat, testing
8 GB	llama3.2:7b, mistral, gemma2:2b	General chat
12 GB	llama3.1:8b, qwen2.5:7b	Better quality
16 GB	llama3.1:70b-q4, codestral	Coding, complex tasks
24 GB+	llama3.1:70b, qwen2.5:72b	Best quality

Use :q4_0 or :q8_0 suffixes for smaller, quantized variants when VRAM is limited.

First Chat and Custom System Prompts

1. Open http://localhost:3000 (or http://YOUR_SERVER_IP:3000).
2. Create an account (first user becomes admin).
3. Settings → Connections — confirm Ollama is connected.
4. Select a model from the dropdown.
5. Start a new chat.

Use Model settings to add custom system prompts (e.g. "You are a helpful coding assistant"). Create RAG documents for context from your files.

Advanced: Multi-User and OpenAI API Compatibility

Multi-user — Open WebUI supports multiple accounts. Admins can manage users, disable sign-ups, and set quotas.

OpenAI-compatible endpoint — Ollama's API is compatible with the OpenAI format. Use:

Base URL: http://localhost:11434/v1
Model: llama3.2

Tools like LangChain, LlamaIndex, and MCP servers can connect to Ollama as if it were OpenAI. Open WebUI can also mix Ollama with real OpenAI/Anthropic in the same interface.

RAG and Documents in Open WebUI

Open WebUI supports RAG (retrieval-augmented generation). Upload PDFs, text files, or point to a folder. Open WebUI chunks and indexes them (via Ollama's embeddings or a separate embed model) and injects relevant context into prompts. Useful for internal docs, support knowledge bases, or personal notes. For larger-scale RAG, see our local RAG chatbot guide with Ollama and ChromaDB, and fine-tune LLaMA 3 for custom models.

Production Deployment

For 24/7 access, deploy on a VPS. Use Docker Compose with both Ollama and Open WebUI, put them behind Traefik or Nginx for HTTPS, and restrict the Open WebUI admin to trusted IPs. For GPU inference at scale, consider vLLM instead of Ollama — it offers higher throughput for concurrent requests. Build a local RAG chatbot for document Q&A, or fine-tune LLaMA 3 for custom domains.

Open WebUI vs LM Studio vs Ollama CLI

Feature	Open WebUI	LM Studio	Ollama CLI
Web interface	Yes	No (desktop app)	No
Multi-user	Yes	No	No
RAG	Yes	Limited	No
Model management	Via Ollama	Built-in	ollama pull/run
Remote access	Yes (server)	Local	SSH
Best for	Server, teams	Local desktop	Terminal users

Choose Open WebUI for a server-hosted, multi-user ChatGPT clone. Use Liquid Web GPU VPS for always-on inference.

Plugins and Extensions

Open WebUI supports plugins for additional functionality: custom models, MCP (Model Context Protocol) integration, and API extensions. Browse the plugin marketplace from Settings → Plugins. For developers, the OpenAI-compatible API makes it easy to build custom tools that call your local Ollama via Open WebUI's proxy. This is useful for RAG pipelines, agents, and automation that need a stable chat endpoint.

Start with a small model

Use ollama pull phi3:mini or llama3.2:3b first. Verify the stack works, then upgrade to larger models. See vLLM production guide for high-throughput serving.

Frequently Asked Questions

Ollama is a local LLM runtime. You run models like LLaMA, Mistral, and Phi with a single command. It exposes an OpenAI-compatible API on port 11434.

Open WebUI is a ChatGPT-like web interface that connects to Ollama, OpenAI, or Anthropic. When used with Ollama, all processing stays on your machine.

Yes. Use small models (3B, 7B quantized). Expect 1–5 tokens per second. For faster inference, use a GPU or cloud GPU VPS.

Run ollama pull <model> from the host or a container that can reach the Ollama API. Models are stored in ~/.ollama (or the container volume).

Ollama's API is OpenAI-compatible. Use base URL http://localhost:11434/v1. Open WebUI can also connect to real OpenAI and Anthropic APIs alongside Ollama.