Firecrawl: guides & tutorials

AI-powered web scrapers use LLMs to replace fragile CSS selectors with natural language extraction prompts. Instead of writing $('.product .price').text(), you instruct the model: "Extract all products with name, price, and availability status."

The result is scrapers that work across multiple page layouts — but come with higher cost and latency than traditional extraction.

Freshness note: Pricing and features verified March 2026.

The Model Context Protocol (MCP) is an open standard (originated by Anthropic, now community-governed) for connecting AI assistants like Claude (Desktop and Code), ChatGPT, VS Code, and Cursor to external tools. The ecosystem has grown quickly, with thousands of community-built servers now available. This guide covers the ones that are most useful for developers, with setup instructions and honest notes on when each fits.

A custom RAG MCP server gives Claude access to a private knowledge base built from your crawled web content. Firecrawl crawls URLs → markdown → chunk → embed → store in ChromaDB or pgvector. Your MCP server exposes a tool that queries the vector store and returns relevant chunks. Claude uses them to answer questions with your data. Start with Firecrawl.

A web-browsing AI assistant: the user asks a question, Claude decides what to search and scrape, Firecrawl fetches content, and Claude synthesizes the answer. Build it with the Anthropic API, Firecrawl Python SDK, and a tool-calling loop. Get Firecrawl for Claude.

The Firecrawl Node.js SDK (@mendable/firecrawl-js) turns any URL into markdown, HTML, or structured JSON with one call. Install it, pass your API key, and use scrape(), crawl(), map(), or extract() — no browser setup required. Get your Firecrawl API key.

Both Firecrawl and Jina Reader convert URLs to clean Markdown for LLMs — but their scope, pricing, and capabilities differ significantly.

TL;DR: Jina Reader is free and instant for single-URL lookups. Firecrawl is the production choice for full-site crawls, structured extraction, and RAG pipelines.

Technical documentation is best consumed by Claude as clean markdown — no HTML tags, nav bars, or code-language headers. Firecrawl's crawl endpoint turns entire doc sites into LLM-ready markdown in minutes. Try Firecrawl for docs.

Most LangChain “web-aware agents” fail because ingestion quality is weak and retrieval context is noisy. This guide shows how to connect Firecrawl and LangChain with a cleaner ingest → index → retrieve loop.

Start here: Firecrawl resources

The best Firecrawl alternatives depend on your needs: Apify for production pipelines, Crawl4AI for self-hosted control, Jina Reader for free single-page conversion, ScrapingBee for raw HTML and proxies, Diffbot for entity extraction, and Bright Data for enterprise-scale collection.

Try Firecrawl →

This guide walks the REST API you will actually call in production. Firecrawl’s base URL is https://api.firecrawl.dev/v2, centered on /scrape, /crawl, and /map. Authenticate with a Bearer token and JSON request bodies. Budget 1 credit per scraped page and 1 credit per map call (same cost no matter how many links you get back).

Get your API key →