Beginner Web Scraping with Claude & MCP in 2026

Contains affiliate links. Disclosure.

You can scrape websites from Claude using MCP servers. No custom code needed. Firecrawl and Apify MCP servers add scrape, crawl, and extraction tools directly into Claude Desktop. Configure once, then ask Claude to fetch pages, extract data, or search the web.

Get Firecrawl API key · Try Apify free

What Is Web Scraping?

Web scraping is automated collection of data from websites. Instead of copying text manually, a tool fetches a page, parses its structure, and returns structured data (tables, product info, articles). Claude with MCP does this by calling external APIs through the Model Context Protocol.

What Is MCP and Why It Matters for Scraping

MCP (Model Context Protocol) is an open standard that lets AI assistants like Claude call external tools. Without MCP, Claude cannot fetch live web content. With an MCP server that exposes scrape tools, Claude gains the ability to pull URLs, crawl domains, and extract data—all from natural language prompts. MCP is supported by Anthropic, OpenAI, and Google in 2026.

Firecrawl MCP + Claude Desktop

Firecrawl provides LLM-ready scraping. It converts pages to clean Markdown, handles JavaScript rendering, and supports batch operations. The Firecrawl MCP server exposes these capabilities inside Claude.

Steps to add Firecrawl MCP:

1. Get an API key at firecrawl.dev/app/api-keys.
2. Open Claude Desktop config. Paths:
   • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
   • Windows: %APPDATA%\Claude\claude_desktop_config.json
   • Linux: ~/.config/Claude/claude_desktop_config.json
3. Add this block (create the file if it does not exist):

{
  "mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": {
        "FIRECRAWL_API_KEY": "fc-YOUR_API_KEY_HERE"
      }
    }
  }
}

4. Replace fc-YOUR_API_KEY_HERE with your real key.
5. Quit Claude Desktop and reopen it.
6. Confirm the tools icon (🔨) appears in the bottom-right.

Your first scrape: ask Claude "Scrape the main content from https://example.com and summarize it."

Apify MCP + Claude Desktop

Apify offers 30,000+ pre-built Actors for sites like Google Maps, Instagram, and Amazon. The Apify MCP server lets Claude run these Actors from chat.

Steps to add Apify MCP:

1. Create an account at apify.com and copy your API token from Settings → Integrations.
2. Edit the same claude_desktop_config.json and add the Apify server:

{
  "mcpServers": {
    "firecrawl": {
      "command": "npx",
      "args": ["-y", "firecrawl-mcp"],
      "env": { "FIRECRAWL_API_KEY": "fc-YOUR_KEY" }
    },
    "apify": {
      "command": "npx",
      "args": ["-y", "@apify/mcp-server-apify"],
      "env": {
        "APIFY_API_TOKEN": "apify_api_YOUR_TOKEN"
      }
    }
  }
}

3. Restart Claude Desktop.
4. Ask Claude to run an Actor, e.g. "Use the Web Scraper Actor to scrape https://example.com".

Common Beginner Mistakes

• Not fully restarting Claude. A config change only takes effect after a complete quit and reopen.
• Wrong config path. Claude reads claude_desktop_config.json, not a different filename.
• Missing Node.js. MCP servers use npx; install Node.js 18+ if you see "npx: command not found."
• Exceeding free limits. Firecrawl free tier: 500 credits/month. Apify free plan: $5 in credits. Test with small runs first.
• Unclear prompts. Be specific: "Scrape the product table from this URL" beats "Get data from the site."

What to Learn Next

Once you can scrape from Claude, deepen your skills:

1. Firecrawl MCP server setup — advanced config for Cursor, VS Code, and Claude.
2. Make.com Claude Desktop MCP — connect Claude to automation workflows.
3. Best Udemy web scraping courses — learn Python and scraping fundamentals for custom tools.

Get Started

Firecrawl gives 500 free credits for clean Markdown output. Apify offers $5/month free credits for 30,000+ Store Actors. Pick one and run your first scrape in Claude.

Pricing verified June 2026. Plans change frequently — check the vendor’s official pricing page for current rates.

Frequently Asked Questions

No. Firecrawl and Apify MCP servers expose tools Claude can call. You ask in plain language; Claude runs the scrape.

Firecrawl suits LLM-style content (Markdown, RAG). Apify suits structured data from popular sites (Google Maps, Instagram). Many users run both.

MCP connects Claude to external APIs. Scraping ethics and legality depend on the target site and your use case—check robots.txt and terms of service.

Ensure you restarted Claude after editing claude_desktop_config.json. Verify the JSON is valid and the API key is set in the env block.