Skip to main content

Octoparse Tutorial: Build Your First Web Scraper in 10 Minutes (2026)

· 7 min read
Yassine El Haddad
Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

This tutorial walks you through building your first Octoparse scraper from scratch. No code. Get Octoparse free or try Apify if you prefer developer tooling. By the end, you'll have a working scraper that extracts data and exports to CSV.

Prerequisites

  • Octoparse account — Free tier works; sign up.
  • Windows PC — The visual builder is Windows-only. Mac users need a VM or cloud Windows.
  • Chrome — Octoparse uses Chromium; having Chrome installed helps.

Step 1: Create New Task and Enter URL

  1. Open Octoparse and log in.
  2. Click NewAdvanced Mode (or use a template for pre-built logic).
  3. Enter the starting URL in the address bar. Example: a product listing page, directory, or job board.
  4. Click Start — Octoparse loads the page in its built-in browser.

Tip: Start with a simple, static page. Avoid JavaScript-heavy SPAs for your first run.

Step 2: Select Data Fields

  1. Click on the first data element you want (e.g., product title, business name).
  2. Octoparse highlights similar elements. Use Auto-detect if it suggests a list.
  3. Confirm the selection covers the repeating list (e.g., all product cards).
  4. Add more fields: click price, rating, URL, etc. Each becomes a column.
  5. Name fields clearly (e.g., title, price, link).
  6. Set data types where relevant (text, number, link).

Common mistake: Selecting a unique element (e.g., page header) instead of a repeating one. Auto-detect helps avoid this.

Step 3: Handle Pagination

If the site has multiple pages:

Click-based pagination (Next button):

  1. Click the Next or page-number link.
  2. Octoparse adds a loop: go to next page → extract → repeat.
  3. Set max pages if you want to limit the run.

URL-based pagination (?page=2):

  1. In the workflow, add a Loop step.
  2. Configure the URL pattern (e.g., replace page=1 with page=[1-N]).
  3. Set the range.

Infinite scroll:

  1. Enable Scroll down to load more data.
  2. Set scroll count or delay between scrolls.
  3. Test on a small run first—infinite scroll can be slow.

Common mistake: Forgetting pagination entirely. You'll only get page one. Always check if the site has more results.

Step 4: Run the Task

  1. Click Run.
  2. Choose Local (free, runs on your PC) or Cloud (paid, runs on Octoparse servers).
  3. Local runs stop when you close the app. Cloud runs continue in the background.
  4. Monitor progress in the run log. Wait for completion.

Local: Good for testing. Cloud: Required for scheduling and IP rotation. On free, only local is available—your PC must stay on. Paid plans unlock cloud; choose Cloud when you run the task.

Debugging Your First Run

If extraction returns empty or wrong data: (1) Re-run Auto-detect—the page structure may have changed. (2) Check that you selected a repeating element, not a header or footer. (3) For pagination, verify the Next button or URL pattern is correct. (4) Add a delay between actions if the site is slow to load. The run log shows each step—use it to see where the workflow fails.

Step 5: Export Data

  1. When the run finishes, go to Data tab.
  2. Export → choose format: CSV, Excel, JSON, Google Sheets, or database (MySQL, MongoDB).
  3. Free plan: CSV, Excel, JSON. Paid plans: more options.
  4. Download or push to cloud storage.

Limit: Free plan caps exports at 10,000 records. See Octoparse Free Plan Guide.

Common Beginner Mistakes

MistakeFix
Not selecting paginationAdd pagination step; otherwise you only get page 1.
Ignoring infinite scrollEnable "Scroll to load more" and set scroll count/delay.
Unstable CSS selectorsUse Auto-detect; avoid brittle class names that change.
Running on JS-heavy siteStart with static pages; use cloud + IP rotation for SPAs.
Too many concurrent local tasksFree plan allows 2; don't overload.

What to Try Next

  • Templates: Browse Octoparse's 100+ templates. Pick one for Amazon, Google Maps, Indeed, etc.
  • Scheduling: On Standard+, set cloud runs on a schedule (daily, weekly).
  • IP rotation: Enable on cloud for anti-bot sites.
  • Advanced workflow: Add loops, conditions, and multi-step navigation.

For Google Maps specifically, see How to Scrape Google Maps with Octoparse. For a full platform verdict, Octoparse Review 2026.

Choosing Your First Target Site

Pick a site with clear structure: repeating list items, simple pagination, minimal JavaScript. Good starters: a product listing page (e.g., a category on a small e-commerce site), a directory (restaurants, services), or a job board. Avoid sites with heavy anti-bot measures (LinkedIn, Amazon at scale) or complex infinite scroll for your first run.

Data Quality Tips

  • Verify field mapping: After the first run, spot-check 10–20 rows. Ensure titles, prices, and URLs are correct.
  • Handle missing values: Some listings may not have every field (e.g., no phone number). Octoparse typically leaves these blank. Plan for nulls in your downstream analysis.
  • Deduplication: If pagination overlaps or the same item appears on multiple pages, dedupe by a unique ID (URL, SKU) before analysis.

When to Switch to Cloud

Local runs are fine for testing. Switch to cloud when: (1) you need scheduling, (2) the run takes >30 minutes and you can't keep the app open, (3) the target site blocks your home IP. Cloud runs use Octoparse's infrastructure and rotating IPs—Standard plan and above.

Apify Affiliate Banner 728x90Apify Affiliate Banner 728x90Apify Affiliate Banner 300x50Apify Affiliate Banner 300x50
Next step

Download Octoparse and run through these steps on a simple directory or product page. If you need custom logic or API integration, Apify offers 6,000+ Actors with full code control.

Frequently Asked Questions

Most users build a basic scraper in under 10 minutes. Mastery (pagination, templates, cloud) takes a few hours of practice.

The visual builder is Windows-only. You can use the cloud dashboard on Mac to run and manage tasks, but building scrapers requires Windows or a VM.

Enable 'Scroll down to load more data' in the workflow. Set scroll count and delay. Test on a small run first; infinite scroll can be slow on complex sites.

Use Auto-detect to re-identify the list. Ensure you selected a repeating element, not a unique one. Check that the page structure hasn't changed.

Common mistakes and fixes

Data not extracting correctly

Use Auto-detect to re-identify the list structure. Ensure you selected a repeating element, not a unique one.

Pagination not working

Explicitly add a pagination step: click Next or configure URL pattern. For infinite scroll, enable scroll-to-load.