Proxies: guides & tutorials

Bright Data and ScraperAPI both solve the same core problem: getting your scraper past anti-bot systems. But they solve it very differently.

ScraperAPI is a lightweight proxy pass-through API. Send a URL, get HTML back. Simple, cheap at small scale, and you own the scraper logic.

Bright Data is an enterprise proxy network plus managed datasets and a cloud browser. More powerful unblocking, more features, higher price tag.

This is a split-decision comparison. Neither is universally "better" — it depends on your volume, target sites, and budget.

ScraperAPI is a proxy rendering API. You send it a URL, it returns HTML routed through their managed proxy pool. Simple, focused, and useful for a specific job: adding a reliable proxy layer to code you already have.

But "simple" doesn't mean "best for everything." This review covers what ScraperAPI actually does, what it costs, where it shines, and where it falls short.

Most proxy vendors make you choose: raw IPs at good prices from a scrappy provider, or a polished dashboard from an enterprise vendor charging three times more. Proxy-Seller has spent a decade building a third option: all five proxy classes (datacenter IPv4/IPv6, ISP, residential, mobile) under one account, at prices that stay competitive even against single-category specialists, with compliance certs that survive legal review.

They've been running since 2014. 500,000+ clients. Every IP exclusively yours, never shared. The residential pool spans 20M+ IPs across 220+ countries.

Is it the right pick for your stack? That depends on whether you want raw IPs you control, or a managed extraction layer someone else runs. This review walks through verified pricing for all five proxy types, the three discount mechanisms that stack, where Proxy-Seller is genuinely strong, and two scenarios where a different provider will serve you better.

Rotating proxies automatically assign a new IP address per request or per session, preventing sites from rate-limiting a single IP. They're the foundation of any production web scraper.

This guide ranks eight major providers by the metrics that matter most: price per GB, IP pool size, bypass success rate on major sites, and support quality.

Claude API applications that scrape data at scale hit rate limits and blocks. Bright Data proxies provide unblockable infrastructure: residential, datacenter, and ISP options that integrate with Python requests or httpx. If you are just getting started, you can try Claude free for a week and grab an Anthropic API key, then try Bright Data for AI pipelines.

Cloudflare Bot Management (including Turnstile, Bot Score, and Managed Rules) is the most common blocker scrapers hit in 2026. It combines TLS fingerprinting, JavaScript challenges, behavioral analysis, and IP reputation scoring — none of which raw requests or fetch can handle.

This guide ranks every bypass method by effectiveness, complexity, and cost.

Legal note: Only scrape data you have a legitimate reason to access. Cloudflare protection is the site's choice; bypassing it may violate ToS and in some jurisdictions, the CFAA. Always check robots.txt and review terms before scraping.

Claude research agents need fresh web data to answer questions, summarize articles, and compare products. Many target sites block datacenter IPs and headless browsers within seconds. IPRoyal residential proxies route traffic through real ISP-assigned IPs, dramatically lowering block rates. This guide shows how to integrate IPRoyal with Python and build a Claude research tool that fetches web content reliably. For full-stack scraping pipelines, pair with Apify.

IPRoyal provides residential rotating proxies with a 32M+ IP pool, starting at approximately $7/GB for small purchases and scaling to ~$1.75/GB at 500 GB — with bandwidth that never expires. This guide covers the exact configuration for Python, Node.js, Playwright, and Crawlee.

Freshness note: Endpoint and format verified March 2026. Check IPRoyal dashboard for current credentials format.

Proxy rotation is the backbone of production web scraping. Single-IP requests get rate-limited, banned, or geo-blocked. Rotating through a pool of IPs spreads load and evades per-IP limits. This technical reference covers per-request rotation, sticky sessions, session pools, geo-targeting, implementation with Bright Data and IPRoyal, Crawlee ProxyConfiguration, backoff strategies, and a comparison table for strategy × use case × provider.

Choosing the wrong proxy type wastes budget and kills pipelines. Datacenter IPs on Instagram get blocked in seconds. Residential proxies on public APIs cost 80x more than necessary. This guide explains all four proxy types (datacenter, residential, ISP, and mobile) with overview tables, use-case mapping, rotating versus sticky sessions, geo-targeting, and provider comparison. For Apify users, see Apify proxy configuration guide. For anti-bot heavy targets, Bright Data Scraping Browser bundles proxies with fingerprint bypass.