Skip to main content

WireGuard VPN for Scraping: Secure Your VPS Traffic

· 9 min read
Yassine El Haddad
Yassine El Haddad
Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

Running a scraping server on a public VPS without a VPN exposes your admin panel, Playwright cluster, and dataset store to the open internet. WireGuard solves this with a minimal kernel-level tunnel that routes all management traffic through an encrypted private channel — no IPsec complexity, no OpenVPN overhead.

This guide walks through installing and configuring WireGuard on a Liquid Web Ubuntu 22.04 VPS, generating keys, routing scraping-server traffic through the tunnel, and locking dashboards behind VPN-only access.

Should you use a VPN for web scraping?​

A VPN on your scraping server is not for hiding your scrapers from target sites — that is what residential proxies are for. The VPN serves a different purpose: securing the administrative plane.

ThreatWithout VPNWith WireGuard VPN
SSH brute-force on port 22Exposed to public internetPort closed; only VPN peers connect
Exposed Playwright/Puppeteer remote portsAny IP can probeBound to wg0 interface only
Scrapy dashboard / Flower / RedisReachable via public IPPrivate to VPN subnet
Inter-worker traffic (multi-server)Plaintext or manual firewallEncrypted mesh over VPN
Team access to job logs / artifactsSSH tunnels or bastion hopsDirect access over VPN subnet

Use WireGuard when:

  • You manage a scraping VPS directly (not via a managed platform like Apify).
  • Multiple developers or workers need access to the same server.
  • You expose any internal dashboard (Scrapyd, Flower, Jupyter, Grafana).

Skip WireGuard when:

  • You delegate all infra to a managed platform and never SSH in directly.
  • The server handles only outbound scraping traffic and has no admin UI.

What is WireGuard?​

WireGuard is a modern VPN protocol built into the Linux kernel since 5.6. It replaces OpenVPN and IPsec in scraping infrastructure for three reasons:

  1. Kernel-level performance. Cryptography runs inside the kernel, not userspace, cutting handshake latency and CPU overhead on high-throughput scraping hosts.
  2. Minimal attack surface. The entire codebase is ~4,000 lines — compared to ~100,000 for OpenVPN. Fewer lines means fewer vulnerabilities.
  3. Zero-config roaming. Client IPs can change (laptop, home network, office) without renegotiating sessions. Useful for development machines hitting a shared scraping server.

Architecture for a scraping stack​

Developer laptop (WireGuard peer)
  ↕ encrypted UDP/51820
Liquid Web VPS (WireGuard server, wg0: 10.8.0.1)
  ├── Playwright cluster (port 9222, bound to wg0)
  ├── Redis (port 6379, bound to 10.8.0.1)
  ├── Grafana dashboard (port 3000, bound to 10.8.0.1)
  └── SSH (port 22, firewall-locked to VPN subnet)

Step 1: Provision a Liquid Web VPS​

Liquid Web VPS plans start with Ubuntu 22.04 LTS and include full root access, which WireGuard requires. For a scraping workload, the 2 vCPU / 2 GB RAM tier handles light-to-medium workloads; upgrade to 4 vCPU / 8 GB for Playwright-based headless jobs.

After provisioning, SSH in as root:

ssh root@YOUR_SERVER_IP

Step 2: Install WireGuard​

WireGuard ships in Ubuntu 22.04's default package repository:

apt update && apt install -y wireguard

Verify the kernel module loads:

modprobe wireguard && lsmod | grep wireguard

Step 3: Generate server key pair​

cd /etc/wireguard
wg genkey | tee server_private.key | wg pubkey > server_public.key
chmod 600 server_private.key

Read both values — you will need them in the server config:

cat server_private.key
cat server_public.key

Step 4: Create the server configuration​

Create /etc/wireguard/wg0.conf:

[Interface]
Address = 10.8.0.1/24
ListenPort = 51820
PrivateKey = <SERVER_PRIVATE_KEY>

# Enable IP forwarding for VPN-only routing
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
PostDown = iptables -D FORWARD -i wg0 -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

# Peer block (one per developer or worker machine)
[Peer]
PublicKey = <CLIENT_PUBLIC_KEY>
AllowedIPs = 10.8.0.2/32

Key config decisions for scraping infrastructure:

  • Address = 10.8.0.1/24 — Assigns the server the .1 address in a /24 subnet. Clients will receive .2, .3, etc.
  • ListenPort = 51820 — Default WireGuard UDP port. Open this in your Liquid Web firewall rules.
  • PostUp/PostDown — iptables rules masquerade outbound traffic from VPN peers through the server's eth0. Required if peers route all traffic through the VPN.
  • AllowedIPs = 10.8.0.2/32 — Restricts this peer to its assigned IP only. Use 0.0.0.0/0 to route all peer traffic through the server (full tunnel mode).

Start and enable the tunnel:

systemctl enable --now wg-quick@wg0
wg show

Step 5: Generate client key pair​

Run on the developer laptop (Linux/macOS) or a second VPS worker:

wg genkey | tee client_private.key | wg pubkey > client_public.key
cat client_public.key  # paste into server's [Peer] block

Step 6: Create the client configuration​

Save as /etc/wireguard/wg0.conf on the client machine:

[Interface]
Address = 10.8.0.2/24
PrivateKey = <CLIENT_PRIVATE_KEY>
DNS = 1.1.1.1

[Peer]
PublicKey = <SERVER_PUBLIC_KEY>
Endpoint = YOUR_SERVER_IP:51820
AllowedIPs = 10.8.0.0/24
PersistentKeepalive = 25
  • AllowedIPs = 10.8.0.0/24 — Routes only VPN-subnet traffic through the tunnel. Scraping traffic still exits the client's local network (not through the server). Use 0.0.0.0/0 to force all traffic through the server.
  • PersistentKeepalive = 25 — Sends a keepalive packet every 25 seconds so NAT sessions do not expire. Essential for laptops behind home routers.

Bring the tunnel up:

wg-quick up wg0
ping 10.8.0.1  # should reach the server

Step 7: Lock services to the VPN interface​

After the tunnel is up, rebind your scraping services to the VPN subnet instead of 0.0.0.0:

Redis​

Edit /etc/redis/redis.conf:

bind 10.8.0.1 127.0.0.1

Playwright remote debugging​

When launching Playwright workers, bind the remote interface to the VPN IP:

chromium --remote-debugging-address=10.8.0.1 --remote-debugging-port=9222

Grafana​

In /etc/grafana/grafana.ini:

[server]
http_addr = 10.8.0.1
http_port = 3000

After rebinding, confirm the service is unreachable from the public internet and reachable from the VPN subnet:

# From developer laptop (connected via WireGuard):
curl http://10.8.0.1:3000  # should respond
curl http://YOUR_SERVER_PUBLIC_IP:3000  # should time out

Step 8: Harden SSH​

Move SSH off the public internet entirely by restricting it to the VPN subnet in /etc/ssh/sshd_config:

ListenAddress 10.8.0.1
ListenAddress 127.0.0.1

Then update your firewall to block port 22 on the public interface:

ufw deny 22/tcp
ufw allow 51820/udp  # WireGuard still reachable from public IP
ufw enable

After this change, SSH access requires an active WireGuard connection. If you lose VPN connectivity, use the Liquid Web out-of-band console in the control panel to recover.

Step 9: Add worker peers (multi-machine scraping)​

For teams or multi-node scraping fleets, each worker gets its own [Peer] block in the server config. Generate a key pair on each worker and add:

# In /etc/wireguard/wg0.conf on the server
[Peer]
# Worker node 1
PublicKey = <WORKER1_PUBLIC_KEY>
AllowedIPs = 10.8.0.3/32

[Peer]
# Worker node 2
PublicKey = <WORKER2_PUBLIC_KEY>
AllowedIPs = 10.8.0.4/32

Reload without dropping existing connections:

wg syncconf wg0 <(wg-quick strip wg0)

Workers on 10.8.0.3 and 10.8.0.4 can now access Redis, the job queue, and artifact storage on 10.8.0.1 without any of those ports exposed publicly.

Troubleshooting​

SymptomLikely causeFix
ping 10.8.0.1 fails from clientFirewall blocks UDP 51820Open UDP 51820 on the Liquid Web server firewall
Handshake shows 0 bytes receivedWrong public key in [Peer] blockRegenerate and verify key pairs
Services unreachable after rebindingServices not restartedsystemctl restart redis grafana-server
SSH locked out after hardeningVPN not connected before SSH lockdownUse Liquid Web out-of-band console
Traffic not routed through VPNMissing PostUp masquerade ruleConfirm iptables -t nat -L shows MASQUERADE rule
Roaming client loses connectionPersistentKeepalive missingAdd PersistentKeepalive = 25 to client config

FAQ​

Should I use a VPN for web scraping?

Use a VPN on your scraping server to secure the admin plane — not to hide scraping traffic from target sites. For evading anti-bot systems, you need rotating residential proxies. WireGuard secures SSH, dashboards, and inter-worker communication; it does not change how target sites see your scraper's IP.

What is WireGuard?

WireGuard is a lightweight VPN protocol built into the Linux kernel since version 5.6. It uses modern cryptography (ChaCha20, Curve25519, BLAKE2) and exposes a simple network interface (wg0) you manage like any other network interface. It outperforms OpenVPN in handshake speed and CPU efficiency on server workloads.

How do I access my scraping server securely?

The safest pattern: install WireGuard on the server, lock all admin services (SSH, Redis, Playwright, dashboards) to the VPN subnet (10.8.0.x), and connect from your laptop or CI machine via a WireGuard client before accessing anything. Liquid Web's out-of-band console provides emergency access if you get locked out.

Does WireGuard work on Windows scraping machines?

Yes. The official WireGuard Windows client imports the same wg0.conf format. Generate the client key pair in PowerShell using the bundled wg.exe binary:

# Run in PowerShell as Administrator after installing WireGuard
cd "C:\Program Files\WireGuard"
.\wg.exe genkey | Tee-Object -Variable private | .\wg.exe pubkey
# $private contains the private key; the output line is the public key
# Save private key to client config; paste public key into server's [Peer] block

Add the peer block to the server config identically to Linux.

Can WireGuard replace my proxy provider?

No. WireGuard routes traffic through your own VPS IP — a single datacenter IP that target sites can fingerprint and block. Proxy providers offer rotating pools of residential and mobile IPs that are statistically indistinguishable from real users. Use WireGuard for infrastructure security; use proxy services for anti-bot evasion.

Setting up a Liquid Web VPS​

Liquid Web is a straightforward choice for scraping VPS workloads: managed Ubuntu images, full root access, and a control panel with out-of-band console access. The out-of-band console is specifically important when hardening SSH behind WireGuard — it provides a recovery path if you accidentally lock yourself out.

For scraping use cases, the managed VPS tier (2–8 vCPU, SSD-backed, unmetered inbound bandwidth) covers most workloads without requiring a dedicated server until you scale to hundreds of concurrent browser sessions.

See the self-hosting guide for the full architecture behind a production scraping stack.