Skip to main content

How to Run Playwright Scrapers on a VPS

· 4 min read
Yassine El Haddad
Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

Running Playwright on a VPS requires Node.js, Chromium and its system dependencies, and a headless config. This guide covers setup on Ubuntu/Debian (e.g. Liquid Web or any provider).

Prerequisites

  • VPS with Ubuntu 22.04 or 24.04 (or similar)
  • Minimum 2 GB RAM — Chromium needs ~1 GB per instance
  • 4 GB RAM recommended — For 2+ concurrent browsers or stability

Step 1: Install Node.js

curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -
sudo apt-get install -y nodejs
node --version # v20.x

Step 2: Install Playwright and Dependencies

mkdir -p ~/scraper && cd ~/scraper
npm init -y
npm install playwright

# Install Chromium browser
npx playwright install chromium

# Install system dependencies (libs Chromium needs)
npx playwright install-deps chromium

On Ubuntu 24.04, if install-deps fails without sudo:

sudo $(which playwright) install-deps chromium

Or use the venv path:

sudo /home/user/scraper/node_modules/.bin/playwright install-deps chromium

Step 3: Minimal Scraper

// scraper.js
const { chromium } = require('playwright');

(async () => {
const browser = await chromium.launch({
headless: true,
args: ['--no-sandbox', '--disable-setuid-sandbox']
});
const page = await browser.newPage();
await page.goto('https://example.com');
const title = await page.title();
console.log(title);
await browser.close();
})();

Run:

node scraper.js

Step 4: Memory Management

  • One browser at a time — Close each browser when done.
  • Limit concurrent pages — Don't open 20+ pages in one browser on 2 GB RAM.
  • --single-process — Reduces memory on small VPS (older Chromium flag; test first).
const browser = await chromium.launch({
headless: true,
args: ['--no-sandbox', '--disable-dev-shm-usage']
});

--disable-dev-shm-usage helps when /dev/shm is small.

Step 5: Process Management with PM2

For long-running or periodic scrapers:

npm install -g pm2
pm2 start scraper.js --name scraper
pm2 startup # Enable on reboot
pm2 save

Or use cron for scheduled runs:

crontab -e
0 * * * * cd /home/user/scraper && node scraper.js >> scraper.log 2>&1

Common System Dependencies (Ubuntu)

If Playwright complains about missing libs:

# Ubuntu/Debian
sudo apt-get install -y \
libnss3 libnspr4 libatk1.0-0 libatk-bridge2.0-0 libcups2 \
libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 \
libxrandr2 libgbm1 libasound2 libpango-1.0-0 libcairo2

Or rely on:

npx playwright install-deps

Alternative: Docker

Avoid dependency issues by using the official Playwright image. See Docker web scraping on Liquid Web.

When to Use Cloud Scraping

For anti-bot targets or zero server ops, use Apify or Firecrawl instead of self-hosting Playwright.

Apify Affiliate Banner 728x90Apify Affiliate Banner 728x90Apify Affiliate Banner 300x50Apify Affiliate Banner 300x50
Next step

Start with 2–4 GB RAM, one browser, and a single target. Scale up only after confirming stability.

Frequently Asked Questions

Yes. Install Node.js, Playwright, Chromium, and system deps (install-deps). Use headless: true. Minimum 2GB RAM.

~1–2 GB per Chromium instance. Allocate 2 GB for one instance, 4 GB for 2–3, 8 GB for heavier concurrency.

OOM: add RAM or limit concurrency. Missing deps: run playwright install-deps. Sandbox: use --no-sandbox in containers.

Common mistakes and fixes

Playwright fails with 'Executable doesn't exist'

Run npx playwright install chromium. For system deps: npx playwright install-deps chromium or sudo $(which playwright) install-deps chromium.

Process killed (OOM) on small VPS

Allocate at least 2GB RAM. Limit concurrent pages. Use --single-process for Chromium in memory-constrained environments.