Skip to main content

Rotating vs Sticky Proxy Sessions: When to Use Each for Scraping

· 11 min read
Yassine El Haddad
Yassine El Haddad
Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

The wrong proxy session type will either get your IP flagged on the first request or corrupt your entire authenticated workflow. Rotating proxies assign a new IP address on every request — ideal for parallel, stateless page scraping. Sticky sessions lock you to one IP for a configurable duration — mandatory when a target site tracks session continuity across requests.

Choosing incorrectly is expensive: rotating sessions break login flows; sticky sessions burn through a single IP until it gets banned on high-volume stateless targets. This guide maps each session type to specific scraping scenarios, with configuration examples for Bright Data and IPRoyal.

What is a rotating proxy session?​

A rotating proxy session assigns a different exit IP on every HTTP request. Your scraper connects to the proxy provider's gateway, and their infrastructure selects a new IP from the pool for each outbound request.

Under the hood: The proxy gateway maps each new TCP connection to a fresh IP address. No session cookie, no state, no continuity between requests.

What it looks like to the target server:

  • Request 1: comes from 185.12.xx.xx (Residential IP, London)
  • Request 2: comes from 91.234.xx.xx (Residential IP, Paris)
  • Request 3: comes from 203.45.xx.xx (Residential IP, Berlin)

The target's rate-limiting system sees three different users, not one bot.

When rotating sessions work​

Scraping taskWhy rotating works
Product catalogue scraping (Amazon, eBay)Each page load is stateless — no link between requests
Google SERP scrapingSearch results require no session continuity
Price monitoring across thousands of URLsHigh parallelism benefits from IP diversity
News article harvestingNo login, no cart, no user state
Public social media profilesNo authentication required

Rotating sessions excel at high-concurrency, stateless scraping. You can open 50 concurrent threads, each using a different IP, and the target sees 50 different visitors.

What is a sticky proxy session?​

A sticky session (also called a persistent session) locks your scraper to a single IP address for a configurable duration — typically 1 to 30 minutes depending on the provider. All requests sent within the session window exit through the same IP.

Under the hood: The proxy gateway assigns a session token (passed in the username credential string) and routes all connections using that token to the same exit node until it expires or you release it.

What it looks like to the target server:

  • Request 1 (GET /login): comes from 185.12.xx.xx
  • Request 2 (POST /login): same IP 185.12.xx.xx
  • Request 3 (GET /account/orders): same IP 185.12.xx.xx

The target sees consistent session continuity — as if a real user navigated across pages.

When sticky sessions are required​

Scraping taskWhy sticky is required
Account login and authenticated scrapingSession cookies are tied to originating IP on many sites
Multi-step checkout flow automationCart state is IP-bound on fraud-sensitive e-commerce sites
Paginated results after a searchSome sites invalidate pagination tokens on IP change
Form submissions with CSRF tokensCSRF validation frequently ties the token to IP + cookie pair
Job board profile scraping (LinkedIn, Indeed)Login sessions fail on IP rotation mid-session

Sticky sessions are mandatory any time the target site links session state to the originating IP address. Rotating mid-session triggers fraud detection or forces re-authentication.

Side-by-side comparison​

DimensionRotating SessionSticky Session
IP assignmentNew IP per requestSame IP for session duration
ConcurrencyHighly parallelizableLimited to IPs in the pool
Anti-ban effectivenessHigh for stateless targetsModerate (single IP burns faster)
Authentication flows❌ Breaks login sessions✅ Maintains session state
Bandwidth costSame as stickySame as rotating
IP ban riskLow (distributed load)Higher (single IP takes all heat)
Best forProduct pages, SERPs, public dataLogins, multi-step flows, checkout
Session durationN/A (per-request)1–30 minutes (provider-configurable)

Configuring rotating vs sticky sessions on Bright Data​

Bright Data controls session type via the username credential string passed to their Super Proxy gateway at brd.superproxy.io:22225.

Rotating session (Python)​

import requests

# Rotating: no session ID in username — new IP per request
# Replace <CUSTOMER_ID> and <PASSWORD> with your Bright Data credentials
proxy_url = "http://brd-customer-<CUSTOMER_ID>-zone-residential:<PASSWORD>@brd.superproxy.io:22225"

proxies = {
    "http": proxy_url,
    "https": proxy_url,
}

for url in product_urls:
    response = requests.get(url, proxies=proxies)
    print(response.status_code)

Each call to requests.get() exits through a different IP automatically.

Sticky session (Python)​

import requests
import uuid

# Sticky: append session ID to username — same IP for session lifetime
# Replace <CUSTOMER_ID> and <PASSWORD> with your Bright Data credentials
session_id = uuid.uuid4().hex  # Generate unique session token

proxy_url = (
    f"http://brd-customer-<CUSTOMER_ID>-zone-residential"
    f"-session-{session_id}:<PASSWORD>@brd.superproxy.io:22225"
)

proxies = {
    "http": proxy_url,
    "https": proxy_url,
}

session = requests.Session()
session.proxies = proxies

# All requests in this session exit through the same IP
session.get("https://example.com/login", data={"user": "x", "pass": "y"})
session.get("https://example.com/account/orders")
session.get("https://example.com/account/invoices")

The -session-{session_id} suffix tells Bright Data's gateway to lock all traffic to a single exit node. Generate a new uuid for each new user session you want to isolate.

Sticky session duration​

Bright Data's default sticky session duration is 10 minutes for residential proxies. You can extend it up to 30 minutes in the zone configuration panel. Beyond 30 minutes, residential nodes may drop the connection due to real device reconnections. For longer workflows, implement session renewal logic:

def get_sticky_proxy(session_id: str) -> dict:
    # Replace <CUSTOMER_ID> and <PASSWORD> with your Bright Data credentials
    proxy_url = (
        f"http://brd-customer-<CUSTOMER_ID>-zone-residential"
        f"-session-{session_id}:<PASSWORD>@brd.superproxy.io:22225"
    )
    return {"http": proxy_url, "https": proxy_url}

# Renew session every 8 minutes for safety margin
import time

session_id = uuid.uuid4().hex
session_start = time.time()

for page_url in paginated_urls:
    if time.time() - session_start > 480:  # 8 minutes
        session_id = uuid.uuid4().hex
        session_start = time.time()
    
    proxies = get_sticky_proxy(session_id)
    response = requests.get(page_url, proxies=proxies)

Configuring rotating vs sticky sessions on IPRoyal​

IPRoyal uses the same credential-string pattern via their proxy gateway at geo.iproyal.com:32325.

Rotating session (Node.js)​

import axios from 'axios';

// Rotating: no session suffix — new IP per request
const proxyConfig = {
  host: 'geo.iproyal.com',
  port: 32325,
  auth: {
    username: 'YOUR_USERNAME',
    password: 'YOUR_PASSWORD',
  },
  protocol: 'http',
};

for (const url of productUrls) {
  const response = await axios.get(url, { proxy: proxyConfig });
  console.log(response.status);
}

Sticky session (Node.js)​

import axios from 'axios';
import { v4 as uuidv4 } from 'uuid';

// Sticky: append _session-{id} to password — same IP for session
const sessionId = uuidv4().replace(/-/g, '').slice(0, 8);

const proxyConfig = {
  host: 'geo.iproyal.com',
  port: 32325,
  auth: {
    username: 'YOUR_USERNAME',
    password: `YOUR_PASSWORD_session-${sessionId}`,
  },
  protocol: 'http',
};

const client = axios.create({ proxy: proxyConfig });

// All requests exit through the same IP
await client.post('https://example.com/login', { user: 'x', pass: 'y' });
const orders = await client.get('https://example.com/account/orders');

IPRoyal's sticky sessions default to 10-minute windows for residential proxies. Non-expiring bandwidth means session failures don't waste purchased GB — only active request bytes count.

Performance implications by scraping scenario​

Scenario 1: Scraping 50,000 product pages from an e-commerce site​

Use rotating sessions.

With rotating sessions, you can run 50 concurrent threads, each assigned a different IP. The target sees 50 simultaneous visitors — well within normal traffic patterns. A single sticky IP making 50,000 sequential requests would hit rate limits within minutes.

Configuration: Use Bright Data's residential pool with rotating sessions. Set concurrency to match your bandwidth budget. No session tokens needed.

Scenario 2: Scraping authenticated account data for 500 business accounts​

Use sticky sessions.

Each account login creates a server-side session tied to the originating IP. If you rotate between the login POST and the subsequent authenticated GETs, the session cookie becomes invalid. Generate one sticky session per account, complete the full login → data extraction → logout flow, then release the session.

Configuration: Generate one session_id per account. Complete the full authentication flow within a single sticky session. Renew the session ID for each new account.

Scenario 3: Scraping paginated search results (50 pages per query)​

Depends on the target site.

Some search engines (DuckDuckGo, Bing) track pagination via URL parameters only — rotating sessions work fine. Others (LinkedIn, Google with personalization enabled) issue a query token tied to the originating IP — sticky sessions prevent token invalidation across pages.

Rule: Test with rotating first. If pages 2+ return empty results or redirect to search, switch to sticky.

Scenario 4: Multi-step form or checkout flows​

Use sticky sessions — always.

CSRF tokens, form tokens, and cart IDs are frequently validated against the originating IP. Even a single IP rotation between form steps will fail CSRF validation and reset the form state. Keep the full form interaction in one sticky session.

How to choose: decision flowchart​

Does the scraping task require login?
  YES → Use sticky session
  NO ↓

Does the task involve multi-step navigation (pagination, form, checkout)?
  YES → Test rotating; if page 2+ fails, switch to sticky
  NO ↓

Does the task involve CSRF tokens or session cookies?
  YES → Use sticky session
  NO ↓

Is maximum concurrency your priority?
  YES → Use rotating session
  NO → Use sticky session (safer default for unknown targets)

Default rule: Start with rotating sessions for any stateless scraping task. Switch to sticky only when authentication, session state, or multi-step flows are involved.

Integrating proxy sessions with Apify actors​

If you run scrapers on Apify, the platform's built-in proxy configuration handles session type through the ProxyConfiguration class in Crawlee:

import { ProxyConfiguration, CheerioCrawler } from 'crawlee';

// Rotating: default behaviour
const rotatingProxy = new ProxyConfiguration({
  proxyUrls: [
    'http://brd-customer-ID-zone-residential:PASS@brd.superproxy.io:22225',
  ],
});

// Sticky: tie session to each request context
const stickyProxy = new ProxyConfiguration({
  proxyUrls: [
    'http://brd-customer-ID-zone-residential-session-SESSIONID:PASS@brd.superproxy.io:22225',
  ],
});

const crawler = new CheerioCrawler({
  proxyConfiguration: rotatingProxy, // or stickyProxy
  async requestHandler({ request, $ }) {
    // extraction logic
  },
});

Apify's SessionPool provides a higher-level abstraction for managing sticky sessions across concurrent requests without manually tracking session IDs. See the proxy rotation guide for the full architecture.

FAQ​

Frequently Asked Questions

A rotating proxy automatically assigns a different IP address to each HTTP request your scraper sends. The proxy provider's gateway selects a new exit node from their IP pool on every connection, making each request appear to originate from a different user. This distributes request volume across many IPs, reducing the chance of any single IP being rate-limited or banned.

You need a sticky (persistent) proxy session whenever your scraping workflow involves state that is tied to the originating IP address. This includes: logging into an account and scraping authenticated pages, navigating multi-step forms with CSRF tokens, paginating through search results where the query token is IP-bound, or scraping checkout flows where cart state is IP-validated. If you rotate IPs mid-session on these targets, the session will be invalidated or flagged as fraudulent.

Keep sticky sessions as short as the task requires. For a login → scrape → logout flow, complete the entire sequence within one session window. Most providers cap residential sticky sessions at 10–30 minutes. Set your session duration to 80% of the maximum to allow for network latency. For Bright Data, the default is 10 minutes (max 30 minutes). For IPRoyal, the default is also 10 minutes. If your workflow exceeds the window, implement session renewal by generating a new session ID and re-authenticating.

Yes. A common pattern is to use rotating sessions for public page scraping and sticky sessions for authenticated sections. For example: discover product URLs on public pages using rotating IPs, then log into a price-tracking account with a sticky session to access member pricing. Manage two separate proxy configurations in your scraper and route requests to the appropriate one based on whether authentication is required.

No. Most proxy providers, including Bright Data and IPRoyal, charge by bandwidth consumed, not by session type. The cost per GB is the same whether you use rotating or sticky sessions. Sticky sessions may consume more bandwidth on high-volume targets because a single IP handles all requests without distribution benefits, potentially triggering rate limiting that forces retries — but the per-GB rate itself is identical.

Start scraping with the right session type​

For most stateless scraping tasks — product pages, search results, public profiles — Bright Data's rotating residential proxies provide the widest IP diversity and best anti-ban coverage at scale. Their 150M+ residential IP pool ensures genuine geographic distribution.

For authenticated scraping or mid-market bandwidth budgets, IPRoyal's sticky session infrastructure pairs non-expiring bandwidth with reliable 10-minute session windows — you pay only for bytes consumed, with no monthly expiration waste.

For teams that want managed orchestration on top of proxy infrastructure, Apify's platform handles session pooling, retry logic, and concurrency management natively — so you configure the proxy type once and let the platform manage session lifecycle.