Firecrawl /map Endpoint: Discover Every URL on a Website

Contains affiliate links. Disclosure.

The Firecrawl /map endpoint discovers URLs on a domain from sitemaps, SERP, and prior crawls. One request costs 1 credit and can return up to 100,000 links. Use it before /crawl to scope extraction and cut waste.

Try Firecrawl map →

What /map Does

POST https://api.firecrawl.dev/v2/map returns a list of URLs with optional title and description. It does not scrape content. Use it for:

• Auditing site structure before crawling
• Building include/exclude path rules
• Creating targeted crawl queues for docs, blogs, or products

Parameters

Parameter	Default	Description
url	required	Base URL to map
limit	5000	Max links (max 100,000)
sitemap	include	skip, include, or only
search	—	Order by keyword relevance (e.g. "blog")
includeSubdomains	true	Include subdomains
ignoreQueryParameters	true	Exclude query-param URLs
ignoreCache	false	Bypass sitemap cache (up to 7 days)
timeout	—	Timeout in milliseconds

Python Example

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR-API-KEY")
mapped = app.map_url(
    "https://example.com",
    limit=5000,
    search="docs",
    sitemap="include"
)
links = mapped.get("links", [])
docs_urls = [l["url"] for l in links if "/docs/" in l["url"] and "/tag/" not in l["url"]]
print(f"Discovered {len(links)} URLs, {len(docs_urls)} in docs")

Response Format

{
  "success": true,
  "links": [
    { "url": "https://example.com/page1", "title": "Page 1", "description": "..." },
    { "url": "https://example.com/page2", "title": null, "description": null }
  ]
}

Each object has url; title and description are optional.

Map → Crawl Workflow

1. Run /map on the target domain
2. Group URLs by path prefix
3. Exclude low-value sections (tags, archives, legal)
4. Build includePaths / excludePaths for /crawl
5. Run /crawl with the scoped config

This reduces crawl spend and improves output quality.

Quality Checks Before Crawl

• Deduplicate mapped URLs
• Canonicalize trailing slash and query params
• Validate domain boundaries (avoid external leaks)
• Store discovery timestamp for freshness

Cost

1 credit per request, regardless of how many links are returned. For large domains, map is far cheaper than blindly crawling.

Next step

Use mapped URLs in batch scraping or RAG ingestion.

Frequently Asked Questions

Yes. Map only discovers URLs; it does not scrape content. One request can return thousands of links for 1 credit.

For unknown domains, yes. It helps build precise path filters and avoid low-value crawl spend.

Normalize URLs, segment by intent, and pass only high-value paths into crawl or scrape workflows.

By default it uses sitemap plus other sources. Set sitemap='only' for sitemap-only, or 'skip' to ignore.