Skip to main content

Firecrawl Structured Extraction: JSON from Any Web Page 2026

· 3 min read
Yassine El Haddad
Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

Firecrawl's /extract endpoint pulls structured data from URLs using a prompt or JSON schema. Provide URLs (with optional wildcards), a schema or prompt, and Firecrawl crawls, parses, and returns collated JSON. Each credit equals ~15 tokens; extraction uses the same credit system as scrape/crawl.

Try Firecrawl extraction →

When to Use Extract​

  • Product data (price, stock, rating)
  • Lead research (company facts, contacts)
  • Content indexing (title, author, date)

Avoid for pages where values are absent or intentionally obfuscated. Use /scrape + custom parsing for fully deterministic output.

Extract with Schema (Python)​

from firecrawl import Firecrawl

app = Firecrawl(api_key="fc-YOUR-API-KEY")
schema = {
"type": "object",
"properties": {
"title": {"type": "string"},
"author": {"type": "string"},
"published_at": {"type": "string"},
"description": {"type": "string"}
},
"required": ["title", "description"]
}

res = app.extract(
urls=["https://example.com/blog/post"],
prompt="Extract the article title, author, publish date, and description.",
schema=schema
)
print(res.get("data", {}))

Extract with Pydantic​

from pydantic import BaseModel
from firecrawl import Firecrawl

class Article(BaseModel):
title: str
author: str | None = None
published_at: str | None = None
canonical_url: str

app = Firecrawl(api_key="fc-YOUR-API-KEY")
res = app.extract(
urls=["https://example.com/blog/post"],
prompt="Extract article metadata.",
schema=Article.model_json_schema()
)
data = res.get("data", {})
Article.model_validate(data) # Post-validate

Extract Without Schema​

Use only a prompt. The model infers structure:

res = app.extract(
urls=["https://example.com/about"],
prompt="Extract the company mission and key features."
)

Use schema when you need stable fields for downstream systems.

Multiple URLs and Wildcards​

res = app.extract(
urls=["https://example.com/docs/*", "https://example.com/blog/post"],
prompt="Extract page title and main topic.",
schema=schema
)

Wildcards (/*) trigger domain crawling. Experimental for large sites; contact support for issues.

Async and Job Status​

job = app.start_extract(
urls=["https://docs.firecrawl.dev/*"],
prompt="Extract company mission and features."
)
job_id = job.get("id")

# Poll until done
status = app.get_extract_status(job_id)
while status.get("status") in ("processing", "pending"):
time.sleep(5)
status = app.get_extract_status(job_id)

data = status.get("data", [])

Results are available for 24 hours after completion.

Accuracy Tips​

  • Define strict schemas with required fields and types
  • Add post-validation before writing to storage
  • Use enable_web_search=True for richer context (costs more)
  • Track field completeness and drift over time
  • Run benchmark URLs when site layouts change

Production Workflow​

  1. URL intake from curated list
  2. Extract with schema
  3. Post-validate (required fields, ranges)
  4. Normalize (currency, dates, locales)
  5. Write to storage with quality metrics
Next step

Apply extraction to ecommerce product data with a product schema.

Frequently Asked Questions

Yes, with a strict schema and post-validation. Without those, reliability drops at scale.

Check required-field presence, type conformance, and range constraints before writing to downstream systems.

Track field-level quality by domain. Re-run benchmark URL sets when layouts change.

Scrape extract is single-page. Extract endpoint supports multiple URLs and wildcards in one job.

Common mistakes and fixes

Extraction returns inconsistent fields

Define a strict schema. Use required fields and type constraints. Add post-validation.

Extract job stuck in processing

Check job status via get_extract_status. Large domains can take minutes. Jobs expire after 24h.

Advertise on this page