Skip to main content

Firecrawl Node.js SDK: Complete JavaScript Developer Guide (2026)

· 7 min read
Yassine El Haddad
Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

The Firecrawl Node.js SDK (@mendable/firecrawl-js) turns any URL into markdown, HTML, or structured JSON with one call. Install it, pass your API key, and use scrape(), crawl(), map(), or extract() — no browser setup required. Get your Firecrawl API key.

Install and initialize

npm install @mendable/firecrawl-js
import Firecrawl from '@mendable/firecrawl-js';

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

Store FIRECRAWL_API_KEY in your environment. Never commit keys. For TypeScript, the package includes types — no @types needed.

scrape(): single page extraction

scrape() fetches one URL and returns content in the formats you specify:

const result = await firecrawl.scrape('https://docs.firecrawl.dev', {
formats: ['markdown', 'html', 'links'],
onlyMainContent: true,
waitFor: 3000,
});
console.log(result.markdown?.slice(0, 500));
console.log(result.links?.length);

Options:

OptionTypeDescription
formatsstring[]markdown, html, json, links, screenshots, extract
includeTagsstring[]CSS selectors for content to include
excludeTagsstring[]CSS selectors for content to exclude
waitFornumberMilliseconds to wait before extraction
onlyMainContentbooleanStrip nav/footer (default: true)

Each page costs 1 Firecrawl credit. Markdown is ideal for RAG; see Firecrawl RAG knowledge base.

crawl(): multi-page extraction

crawl() discovers and scrapes pages starting from a URL. It blocks until the crawl completes (or times out):

const job = await firecrawl.crawl('https://docs.firecrawl.dev', {
limit: 25,
scrapeOptions: {
formats: ['markdown'],
onlyMainContent: true,
},
excludePaths: ['/changelog/*', '/api/*'],
includePaths: ['/docs/*'],
maxDepth: 2,
pollInterval: 2,
timeout: 120,
});

console.log(job.status); // 'completed'
console.log(job.data?.length); // Array of scraped docs

Options:

OptionTypeDescription
limitnumberMax pages to scrape
returnOnlyUrlsbooleanReturn URLs only, no content
maxDepthnumberCrawl depth from start URL
includePathsstring[]Glob patterns for allowed paths
excludePathsstring[]Glob patterns to skip
scrapeOptionsobjectSame as scrape() options
pollIntervalnumberSeconds between status checks
timeoutnumberMax seconds to wait

For non-blocking crawls, use startCrawl() to get a job ID, then getCrawlStatus() or the watcher() for real-time updates.

map(): sitemap-style URL discovery

map() returns URLs on a domain without extracting content. One credit per request regardless of link count:

const res = await firecrawl.map('https://example.com', { limit: 100 });
console.log(res.links); // Array of URL strings

Use map() to scope URLs before a crawl, or to build a sitemap. See Firecrawl API tutorial for the full endpoint strategy.

extract(): LLM-powered structured extraction

extract() pulls structured data using a schema or prompt. Firecrawl uses an internal LLM to populate your JSON schema:

const result = await firecrawl.extract({
urls: ['https://example.com/product/123'],
schema: {
type: 'object',
properties: {
title: { type: 'string' },
price: { type: 'number' },
inStock: { type: 'boolean' },
},
required: ['title', 'price'],
},
prompt: 'Extract the product title, price in USD, and stock availability.',
});
console.log(result.data);

For Zod schemas, convert to JSON schema with zodToJsonSchema. Extraction credits scale with tokens (~15 tokens per credit). See Firecrawl structured extraction for schema best practices.

batchScrape(): multiple URLs

batchScrape() scrapes many URLs in one job. Use it when you have a known URL list (e.g., from map() or a sitemap):

const urls = [
'https://docs.firecrawl.dev/scrape',
'https://docs.firecrawl.dev/crawl',
'https://docs.firecrawl.dev/map',
];

const job = await firecrawl.batchScrape(urls, {
options: {
formats: ['markdown'],
onlyMainContent: true,
},
});

console.log(job.data); // Array of { markdown, metadata, ... } per URL

For large batches, use startBatchScrape() + getBatchScrapeStatus() with autoPaginate to avoid memory issues.

TypeScript and error handling

The SDK ships TypeScript types. Wrap calls in try/catch for production:

import Firecrawl from '@mendable/firecrawl-js';

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY! });

async function scrapeSafe(url: string) {
try {
const result = await firecrawl.scrape(url, { formats: ['markdown'] });
return { ok: true, markdown: result.markdown };
} catch (err: unknown) {
const message = err instanceof Error ? err.message : String(err);
if (message.includes('402')) {
console.error('Credits exhausted');
} else if (message.includes('429')) {
console.error('Rate limited');
}
return { ok: false, error: message };
}
}

Check for 402 Payment Required (credits), 429 Too Many Requests (rate limit), and timeouts. See Firecrawl pricing for tier limits.

Complete example: crawl docs and build a JSON knowledge base

import Firecrawl from '@mendable/firecrawl-js';
import { writeFileSync } from 'fs';

const firecrawl = new Firecrawl({ apiKey: process.env.FIRECRAWL_API_KEY });

async function buildDocsKnowledgeBase(baseUrl) {
const mapped = await firecrawl.map(baseUrl, { limit: 50 });
const urlsToCrawl = mapped.links?.slice(0, 20) ?? [baseUrl];

const job = await firecrawl.crawl(baseUrl, {
limit: 20,
includePaths: urlsToCrawl.map((u) => new URL(u).pathname + '*'),
scrapeOptions: { formats: ['markdown'], onlyMainContent: true },
pollInterval: 2,
timeout: 90,
});

const knowledgeBase = (job.data ?? []).map((doc) => ({
url: doc.metadata?.sourceURL ?? doc.url,
title: doc.metadata?.title ?? '',
content: doc.markdown ?? '',
}));

writeFileSync('docs-kb.json', JSON.stringify(knowledgeBase, null, 2));
return knowledgeBase;
}

buildDocsKnowledgeBase('https://docs.firecrawl.dev').then((kb) =>
console.log(`Saved ${kb.length} pages`)
);

This pattern powers RAG pipelines. Chunk the markdown, embed, and store in Chroma or pgvector — see Firecrawl RAG knowledge base.

Comparison: Node.js SDK vs Python SDK vs REST API

AttributeNode.js SDKPython SDKREST API
Package@mendable/firecrawl-jsfirecrawl-pyfetch / axios
Installnpm installpip installNone
AsyncNative async/awaitFirecrawlAsyncYour HTTP client
Watcherwatcher() for crawlPolling / customWebhooks
TypeScriptBuilt-inType hintsN/A
BatchbatchScrape()Equivalent/batch endpoint
Best forJS/TS stacks, MCP, Node servicesPython pipelines, LangChainCustom integrations

Winner for JavaScript developers: Node.js SDK — first-class async, TypeScript, and MCP server compatibility.

Polling and webhooks

crawl() and batchScrape() poll until completion. For long jobs:

  1. startCrawl() → get id
  2. getCrawlStatus(id) → check status, data, next
  3. Use watcher(id, { kind: 'crawl' }) for real-time document events

Firecrawl also supports webhooks. Configure in the dashboard to receive a POST when a job completes — useful for serverless or event-driven pipelines.

Apify Affiliate Banner 728x90Apify Affiliate Banner 728x90Apify Affiliate Banner 300x50Apify Affiliate Banner 300x50
Start with scrape, scale with crawl

Use scrape() for single pages and ad-hoc extraction. Use crawl() when you need full-site coverage. Map first with map() to scope URLs and control cost.



Try Firecrawl Node.js SDK | Firecrawl MCP for Claude

Frequently Asked Questions

Run npm install @mendable/firecrawl-js. Import with import Firecrawl from '@mendable/firecrawl-js' and initialize with new Firecrawl({ apiKey }).

scrape() extracts one URL. crawl() starts from a URL, discovers linked pages, and scrapes up to a limit. Use scrape for single pages; crawl for full-site extraction.

Yes. The package includes TypeScript definitions. No separate @types package needed.

Use extract() with a JSON schema and optional prompt. Firecrawl uses an LLM to populate the schema from page content. For single-page extraction, use scrape() with formats: ['extract'] and schema.

Yes. Use the Firecrawl MCP server (npx -y firecrawl-mcp) to give Claude and Cursor scrape, crawl, map, and extract tools. See the Firecrawl MCP server setup guide.

Increase the timeout option in crawl(). For very large crawls, use startCrawl() + getCrawlStatus() with manual pagination, or the watcher() for streaming.

Common mistakes and fixes

ModuleNotFoundError or require() of ESM

Use npm install @mendable/firecrawl-js. For CommonJS, use dynamic import() or set type: module in package.json.

Crawl times out or returns partial data

Increase timeout in crawl options. Use pollInterval to check status. For large crawls, use startCrawl + getCrawlStatus with pagination.