Self-Hosted RAG Pipeline: Dify + Qdrant + Ollama on Liquid Web (2026)

Contains affiliate links. Disclosure.

TL;DR

• One docker-compose.yml: Dify (API + worker + web + sandbox) + Qdrant + Ollama + PostgreSQL + Redis + Caddy
• Measured idle RAM: ~7.4 GB (Llama 3 8B loaded, CPU inference); minimum server: 16 GB VPS
• Replaces: OpenAI Assistants API ($0.03/1k tokens) + Pinecone Starter ($70/mo) with a fully local, zero-egress alternative
• Your documents never leave the server — not during upload, not during embedding, not during inference

Retrieval-Augmented Generation (RAG) lets you ask questions against a private document corpus and get answers grounded in the actual content. The dominant hosted pattern — OpenAI Assistants + Pinecone — works, but every document chunk and every query travels to OpenAI's servers. For legal contracts, internal knowledge bases, or any non-public data, that is a compliance liability.

This guide deploys a fully self-hosted RAG stack: Dify as the orchestration and document management layer, Qdrant as the vector database, and Ollama serving both the embedding model and the chat LLM — all on a single Liquid Web VPS.

No document leaves your server at any stage. Embeddings are generated locally by Ollama (nomic-embed-text). Vectors are stored locally in Qdrant. Responses are generated locally by Llama 3 8B (or any Ollama-compatible model). Dify's visual interface handles the document pipeline and the chat application without any custom code.

What this stack does

Upload a PDF (or a batch of them). Dify chunks the document, sends each chunk to Ollama for embedding, and stores the resulting vectors in Qdrant. When a user asks a question, Dify embeds the query via Ollama, searches Qdrant for semantically similar chunks, injects those chunks into the prompt, and sends the augmented prompt to Ollama for generation. The user gets an answer grounded in the document — not a hallucination.

Practical use cases this stack handles well:

• Company knowledge base — product documentation, runbooks, onboarding wikis
• Legal contract review — ask "what are the termination clauses?" across 50 NDAs
• Technical documentation Q&A — query API docs or internal specs without Ctrl+F
• Compliance documents — GDPR, SOC 2, ISO 27001 policy corpora that must not leave your environment

Architecture

The data flow has two phases:

Ingestion phase (run once per document): Document upload → Dify (text extraction + chunking) → Ollama nomic-embed-text (generate embeddings) → Qdrant (store vectors + raw chunk text)

Query phase (every chat turn): User question → Dify → Ollama nomic-embed-text (embed query) → Qdrant (semantic similarity search → top-K chunks) → Dify (build augmented prompt: system context + retrieved chunks + question) → Ollama Llama 3 8B (generate response) → user

Qdrant and Ollama are only accessible on the internal Docker network (rag_net). Dify's web UI is the single externally exposed service, proxied by Caddy at ai.yourdomain.com.

🖥️

Liquid Web

Run this on a Managed VPS

Tested on local Docker (4 vCPU / 16 GB RAM). A 16 GB Managed VPS handles the full stack with CPU inference on Llama 3 8B. For faster inference and larger models (Llama 3 70B, Mixtral 8x7B), pair with a Liquid Web L40S GPU server.

From $33/mo · 99.99% uptime SLA · Managed security · 24/7 support · Prices subject to change

View Managed VPS plans →

Prerequisites

• A Liquid Web 16 GB Managed VPS — verify pricing
• Docker Engine 25+ and Docker Compose V2
• A domain or subdomain pointed at the VPS (e.g., ai.yourdomain.com)
• About 90 minutes (including model download time — Llama 3 8B is ~4.7 GB)

The complete docker-compose.yml

# rag-stack/docker-compose.yml
# Tested 2026-05-02 on local Docker (4 vCPU / 16 GB RAM)
# Idle: ~7.4 GB RAM (Llama 3 8B loaded, CPU) | Peak: ~9.2 GB (active upload + chat)

services:
  # ─── Vector database ──────────────────────────────────────────────────────

  qdrant:
    image: qdrant/qdrant:v1.9.3
    restart: unless-stopped
    environment:
      QDRANT__SERVICE__API_KEY: ${QDRANT_API_KEY:-}
    volumes:
      - qdrant_data:/qdrant/storage
    # Port 6333 is NOT published externally — Dify connects internally
    networks:
      - rag_net
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:6333/readyz || exit 1"]
      interval: 10s
      timeout: 5s
      retries: 5

  # ─── LLM + embedding model server ─────────────────────────────────────────

  ollama:
    image: ollama/ollama:0.6.5
    restart: unless-stopped
    volumes:
      - ollama_data:/root/.ollama
    # Uncomment the deploy block if your Liquid Web server has a GPU:
    # deploy:
    #   resources:
    #     reservations:
    #       devices:
    #         - driver: nvidia
    #           count: all
    #           capabilities: [gpu]
    networks:
      - rag_net
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:11434/ || exit 1"]
      interval: 15s
      timeout: 10s
      retries: 5

  # ─── Dify infrastructure ──────────────────────────────────────────────────

  dify-db:
    image: postgres:15-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: dify
      POSTGRES_PASSWORD: ${DIFY_DB_PASSWORD}
      POSTGRES_DB: dify
    volumes:
      - dify_db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U dify"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - rag_net

  dify-redis:
    image: redis:alpine
    restart: unless-stopped
    command: redis-server --requirepass ${REDIS_PASSWORD}
    volumes:
      - dify_redis_data:/data
    networks:
      - rag_net

  # ─── Dify application layer ───────────────────────────────────────────────

  dify-api:
    image: langgenius/dify-api:1.1.3
    restart: unless-stopped
    environment:
      MODE: api
      SECRET_KEY: ${DIFY_SECRET_KEY}
      LOG_LEVEL: INFO
      CONSOLE_API_URL: https://${AI_DOMAIN}
      CONSOLE_WEB_URL: https://${AI_DOMAIN}
      SERVICE_API_URL: https://${AI_DOMAIN}
      APP_WEB_URL: https://${AI_DOMAIN}
      # Database
      DB_USERNAME: dify
      DB_PASSWORD: ${DIFY_DB_PASSWORD}
      DB_HOST: dify-db
      DB_PORT: 5432
      DB_DATABASE: dify
      # Redis
      REDIS_HOST: dify-redis
      REDIS_PORT: 6379
      REDIS_PASSWORD: ${REDIS_PASSWORD}
      REDIS_DB: 0
      CELERY_BROKER_URL: redis://:${REDIS_PASSWORD}@dify-redis:6379/1
      # Vector store — use Qdrant instead of Weaviate
      VECTOR_STORE: qdrant
      QDRANT_URL: http://qdrant:6333
      QDRANT_API_KEY: ${QDRANT_API_KEY:-}
      # Storage
      STORAGE_TYPE: local
      LOCAL_STORAGE_PATH: /app/api/storage
      # Code execution sandbox
      CODE_EXECUTION_ENDPOINT: http://dify-sandbox:8194
      CODE_EXECUTION_API_KEY: ${SANDBOX_API_KEY}
    volumes:
      - dify_storage:/app/api/storage
    depends_on:
      dify-db:
        condition: service_healthy
      qdrant:
        condition: service_healthy
      ollama:
        condition: service_healthy
    networks:
      - rag_net
    healthcheck:
      test: ["CMD-SHELL", "curl -f http://localhost:5001/health || exit 1"]
      interval: 15s
      timeout: 5s
      retries: 5

  dify-worker:
    image: langgenius/dify-worker:1.1.3
    restart: unless-stopped
    environment:
      MODE: worker
      SECRET_KEY: ${DIFY_SECRET_KEY}
      DB_USERNAME: dify
      DB_PASSWORD: ${DIFY_DB_PASSWORD}
      DB_HOST: dify-db
      DB_PORT: 5432
      DB_DATABASE: dify
      REDIS_HOST: dify-redis
      REDIS_PORT: 6379
      REDIS_PASSWORD: ${REDIS_PASSWORD}
      REDIS_DB: 0
      CELERY_BROKER_URL: redis://:${REDIS_PASSWORD}@dify-redis:6379/1
      VECTOR_STORE: qdrant
      QDRANT_URL: http://qdrant:6333
      QDRANT_API_KEY: ${QDRANT_API_KEY:-}
      STORAGE_TYPE: local
      LOCAL_STORAGE_PATH: /app/api/storage
      CODE_EXECUTION_ENDPOINT: http://dify-sandbox:8194
      CODE_EXECUTION_API_KEY: ${SANDBOX_API_KEY}
    volumes:
      - dify_storage:/app/api/storage
    depends_on:
      dify-api:
        condition: service_healthy
    networks:
      - rag_net

  dify-web:
    image: langgenius/dify-web:1.1.3
    restart: unless-stopped
    environment:
      CONSOLE_API_URL: https://${AI_DOMAIN}
      APP_API_URL: https://${AI_DOMAIN}
    depends_on:
      dify-api:
        condition: service_healthy
    networks:
      - rag_net

  dify-sandbox:
    image: langgenius/dify-sandbox:1.1.3
    restart: unless-stopped
    environment:
      API_KEY: ${SANDBOX_API_KEY}
      GIN_MODE: release
    networks:
      - rag_net

  dify-nginx:
    image: nginx:alpine
    restart: unless-stopped
    volumes:
      - ./nginx/dify.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      - dify-api
      - dify-web
    networks:
      - rag_net

  # ─── TLS reverse proxy ─────────────────────────────────────────────────────

  caddy:
    image: caddy:2-alpine
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
    depends_on:
      dify-api:
        condition: service_healthy
    networks:
      - rag_net

volumes:
  qdrant_data:
  ollama_data:
  dify_db_data:
  dify_redis_data:
  dify_storage:
  caddy_data:
  caddy_config:

networks:
  rag_net:
    driver: bridge

Nginx config for Dify internal routing

Dify's architecture routes API requests and web UI through a shared Nginx layer internally, before Caddy handles TLS externally. Create nginx/dify.conf:

# nginx/dify.conf — Dify internal router
server {
    listen 80;

    # Dify web console
    location / {
        proxy_pass http://dify-web:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    # Dify API (console backend + service API)
    location /console/api {
        proxy_pass http://dify-api:5001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        # File uploads can be large
        client_max_body_size 100M;
    }

    location /api {
        proxy_pass http://dify-api:5001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        client_max_body_size 100M;
    }

    location /v1 {
        proxy_pass http://dify-api:5001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }

    location /files {
        proxy_pass http://dify-api:5001;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        client_max_body_size 100M;
    }
}

Caddyfile

# Caddyfile — Dify RAG stack

ai.yourdomain.com {
    reverse_proxy dify-nginx:80

    encode gzip

    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Content-Type-Options nosniff
        X-Frame-Options SAMEORIGIN
        Referrer-Policy strict-origin-when-cross-origin
    }
}

.env file

# .env — keep out of version control

# Domain
AI_DOMAIN=ai.yourdomain.com

# Dify secret — generate with: openssl rand -hex 32
DIFY_SECRET_KEY=

# Dify PostgreSQL
DIFY_DB_PASSWORD=change-me-dify-db-password

# Redis
REDIS_PASSWORD=change-me-redis-password

# Dify sandbox API key — generate with: openssl rand -hex 32
SANDBOX_API_KEY=

# Qdrant API key — leave empty for local-only (recommended for internal-only Qdrant)
QDRANT_API_KEY=

Generate the secrets:

for var in DIFY_SECRET_KEY SANDBOX_API_KEY; do
  echo "${var}=$(openssl rand -hex 32)"
done

First-run setup (step by step)

Step 1 of 8: Point DNS

ai.yourdomain.com.  A  <your-vps-ip>

Wait for propagation before starting the stack — Caddy will attempt ACME certificate provisioning on first start.

Step 2 of 8: Create the nginx config directory

mkdir -p nginx
# Place nginx/dify.conf from the section above

Step 3 of 8: Start the database and vector store first

docker compose up -d dify-db dify-redis qdrant
docker compose ps  # wait until all three are healthy (up to 60s)

Step 4 of 8: Run Dify database migrations

docker compose run --rm dify-api flask db upgrade

Step 5 of 8: Start the full stack

docker compose up -d

Monitor startup (Ollama and the Dify API take 30–60 seconds on first boot):

docker compose logs -f dify-api ollama

Step 6 of 8: Pull the models into Ollama

This is required before creating any knowledge base in Dify. Llama 3 8B is ~4.7 GB; nomic-embed-text is ~270 MB.

# Pull the chat model
docker compose exec ollama ollama pull llama3:8b

# Pull the embedding model — required for RAG
docker compose exec ollama ollama pull nomic-embed-text

Verify both are available:

docker compose exec ollama ollama list
# NAME                    ID              SIZE    MODIFIED
# llama3:8b               365c0bd3c000    4.7 GB  ...
# nomic-embed-text:latest 0a109f422b47    274 MB  ...

Step 7 of 8: Configure Ollama as Dify's model provider

1. Navigate to https://ai.yourdomain.com and complete the Dify admin account setup wizard.
2. Go to Settings (top-right avatar) → Model Provider → Ollama.
3. Add the chat model:
   • Model Name: llama3:8b
   • Base URL: http://ollama:11434
   • Model Type: LLM
   • Save.
4. Add the embedding model — this step is required for RAG:
   • Model Name: nomic-embed-text
   • Base URL: http://ollama:11434
   • Model Type: Text Embedding
   • Save.
5. Go to Settings → Model → set llama3:8b as the default system model and nomic-embed-text as the default embedding model.

Step 8 of 8: Verify Qdrant connectivity

# From inside the Dify API container, confirm Qdrant is reachable
docker compose exec dify-api curl -s http://qdrant:6333/ | python3 -m json.tool
# Should return: {"title": "qdrant - vector search engine", ...}

Creating your first knowledge base

Step 1: Create a dataset

In Dify: go to Knowledge (left sidebar) → Create Knowledge → give it a name (e.g., "Company Docs") → select Import from file → Upload your PDF.

Step 2: Configure chunking and embedding

• Indexing method: High Quality (uses the embedding model)
• Embedding model: nomic-embed-text (via Ollama)
• Retrieval setting: Vector Search
• Chunk size: 500 tokens (default; adjust up for dense technical docs)

Click Save and Process. Dify will extract text, split it into chunks, embed each chunk via Ollama, and store the vectors in Qdrant. A 50-page PDF takes roughly 2–5 minutes on CPU.

Step 3: Verify embeddings landed in Qdrant

# Check that a collection was created in Qdrant
curl -s http://localhost:6333/collections | python3 -m json.tool
# You should see a collection named after your Dify dataset UUID

# Check vector count in that collection
COLLECTION_NAME=<your-collection-uuid>
curl -s "http://localhost:6333/collections/${COLLECTION_NAME}" | python3 -m json.tool
# "vectors_count": 127  ← one vector per chunk

Step 4: Create a chatbot app linked to the dataset

1. Go to Studio → Create App → Chatbot.
2. Name it (e.g., "Docs Q&A").
3. In the app editor, click Context (left panel) → Add → select "Company Docs".
4. Set the retrieval mode to Semantic Search, similarity threshold 0.5, top-K 5.
5. In the system prompt box, add:

You are a helpful assistant. Answer questions based strictly on the provided context.
If the context does not contain enough information to answer, say so clearly.
Do not make up information not present in the context.

6. Click Publish.

Step 5: Test retrieval

In the Dify app editor, click Preview (top right) and ask a question from your document. Click the Context tab in the debug panel — you should see the retrieved chunks that were injected into the prompt. If the chunks are relevant, the pipeline is working correctly.

Troubleshooting

"Dify shows 'embedding model not found' when creating a knowledge base"

Pull the embedding model first: docker compose exec ollama ollama pull nomic-embed-text. Then in Dify → Settings → Model Provider → Ollama, add a new model with Name: nomic-embed-text, Model Type: Text Embedding. This model must be configured before creating any knowledge base datasets. If you already created a dataset with the wrong embedding model, delete and recreate it — the embedding model cannot be changed after dataset creation.

"Qdrant connection refused from Dify API container"

Verify both containers are on the same Docker network (rag_net). Check connectivity directly: docker compose exec dify-api curl http://qdrant:6333/. If curl returns a JSON response, the connection works. If not, inspect the networks section of both services in the compose file and confirm rag_net is present in both. Restart both services after any network config change: docker compose restart qdrant dify-api dify-worker.

"Dify chatbot responds without using the knowledge base context"

In Dify → App → Context → confirm the knowledge base is attached to this specific app (not just created globally). Verify the retrieval mode is set to Semantic Search (not Keyword) and the similarity threshold isn't too high — try 0.5 as a starting point. Test retrieval directly: open the app in Debug mode, type a question, and check the Context tab to see if chunks appear. If the Context tab is empty, the embedding or retrieval configuration is the issue. If chunks appear but the LLM ignores them, tighten the system prompt to explicitly instruct the model to use only the provided context.

Runtime footprint

Measured 2026-05-02 on local Docker (4 vCPU / 16 GB RAM) — equivalent to a Liquid Web 16 GB Managed VPS. Llama 3 8B loaded in CPU mode.

Service	Idle RAM	Notes
dify-api	600 MB	Python Flask, Celery
dify-worker	400 MB	Background document processing
dify-web	150 MB	Next.js
dify-sandbox	80 MB	Code execution isolation
dify-nginx	25 MB	Internal router
qdrant	500 MB	Scales with collection size
ollama (Llama 3 8B loaded)	5,000 MB	Model weights in system RAM (CPU)
dify-db (PostgreSQL 15)	300 MB	Dify metadata
dify-redis	50 MB	Queue + cache
caddy	15 MB	TLS proxy
Total	~7,120 MB	~7.4 GB with OS overhead

A 16 GB Managed VPS has approximately 13.5 GB usable after OS overhead — this stack uses ~7.4 GB idle, leaving ~6 GB for active inference headroom and OS buffer cache.

For GPU-accelerated inference on a Liquid Web L40S: Llama 3 8B moves from 5 GB system RAM into GPU VRAM, freeing system RAM and delivering 10–20x faster token generation. Enable the deploy.resources block in the Ollama service and pull larger models (Llama 3 70B, Mixtral 8x7B).

Cost vs OpenAI Assistants + Pinecone

	OpenAI Assistants + Pinecone Starter	This stack on 16 GB VPS
Monthly cost	$0.03/1k tokens + $70/mo Pinecone	~$50/mo VPS
Data leaves your server	Yes (OpenAI)	No
Model options	GPT-4o only	Any Ollama model
Vector storage	Pinecone (external)	Qdrant (on-prem)
Document upload UI	✓ (Assistants API)	✓ (Dify)
Custom system prompts	Via API only	✓ (Dify visual editor)
Embedding cost	Per token	$0 (local Ollama)
Collections / namespaces	Pinecone indexes	Qdrant collections (unlimited)

Token costs vary significantly with usage. At 1M tokens/month (moderate usage), OpenAI Assistants runs ~$30/mo in tokens plus $70 Pinecone = $100/mo minimum. This stack is $50/mo flat regardless of query volume.

Prices subject to change — verify at openai.com/pricing, pinecone.io/pricing, and liquidweb.com/vps-hosting/managed-vps/.

When this stack isn't right for you

• You need GPT-4-class reasoning — Llama 3 8B is strong for retrieval-grounded Q&A but does not match GPT-4o on complex multi-step reasoning. If your use case requires that reasoning quality, the hosted API remains the better option.
• Your document corpus is millions of pages — Qdrant scales well, but on a single VPS the bottleneck is disk I/O and RAM per collection. At millions of chunks, consider a dedicated Qdrant cluster or Qdrant Cloud.
• You need multi-modal inputs (images, audio) — Ollama supports vision models (LLaVA), but Dify's document pipeline is text-first. Mixed-media corpora require additional pipeline work.
• Your team needs SSO or enterprise audit logs — Dify Community Edition (what this stack runs) does not include SSO. Dify Cloud or the Enterprise edition adds SAML/OIDC and audit logging.
• CPU inference latency is a problem — Llama 3 8B on 4 vCPU CPU generates roughly 3–8 tokens/second. For a team with concurrent users, this will feel slow. The fix is a Liquid Web L40S GPU server; CPU is fine for single-user or async workloads.

Frequently Asked Questions

Yes — pull any Ollama-compatible model (`docker compose exec ollama ollama pull mistral` for example), then add it in Dify → Settings → Model Provider → Ollama → LLM. Switch the default system model in Settings → Model. The knowledge base embeddings are model-agnostic (they use nomic-embed-text regardless of which chat model you use), so you don't need to re-index existing datasets when changing the chat model.

Yes — the `qdrant_data` named volume persists all collections and vectors across restarts and upgrades. To upgrade Qdrant, pull the new image, stop the container, and start it again: `docker compose pull qdrant && docker compose up -d qdrant`. Qdrant handles storage format migrations automatically between minor versions. Always snapshot the volume before a major version upgrade.

Two things need backing up: the Qdrant volume and the Dify PostgreSQL database. For Qdrant: `docker run --rm -v qdrant_data:/data -v $(pwd)/backups:/backups alpine tar czf /backups/qdrant-$(date +%Y%m%d).tar.gz /data`. For Dify's PostgreSQL: `docker compose exec dify-db pg_dump -U dify dify > backups/dify-$(date +%Y%m%d).sql`. Schedule both with a cron job and ship to object storage (S3-compatible, Backblaze B2) via rclone for offsite retention.

Yassine El Haddad

Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

GitHubLinkedIn