Skip to main content

Make.com + Firecrawl: Automate Web Content Extraction for AI Workflows

· 11 min read
Yassine El Haddad
Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

Yes, Make.com can call the Firecrawl API directly using its built-in HTTP module — no custom code, no infrastructure setup. You configure a POST request to Firecrawl's /scrape endpoint, receive clean markdown back, and pipe that into an OpenAI or Claude module for analysis. The entire pipeline takes under 20 minutes to build.

This tutorial walks through building a daily blog monitoring and AI summary pipeline: Make.com fetches a list of URLs on a schedule, Firecrawl converts each page to LLM-ready markdown, and an AI module generates a structured summary you can deliver to Slack, email, or a Google Sheet.

What You'll Build

A Make.com scenario with four core stages:

  1. Trigger — a schedule module runs daily at 9 AM
  2. HTTP module — calls Firecrawl /scrape for each URL
  3. Iterator + parser — unpacks the markdown response
  4. AI module — sends markdown to OpenAI for summarization

The output is a structured digest: URL, page title, and a 3-sentence AI summary, routed to whatever destination you use for internal reporting.

Prerequisites

RequirementDetails
Make.com accountFree tier works; sign up here
Firecrawl API keyAvailable on the Firecrawl dashboard — free tier includes 500 credits/month
OpenAI or Claude API keyFor the AI summarization step
A list of URLs to monitorCan be hardcoded or pulled from a Google Sheet

No coding required. Make.com's HTTP module handles the REST call natively.

How Make.com HTTP Module Calls Firecrawl

Firecrawl exposes a REST API. Its primary endpoint is:

POST https://api.firecrawl.dev/v1/scrape

The request body specifies the target URL and the output format. Firecrawl handles JavaScript rendering, cookie banners, and anti-bot measures internally — you get clean markdown back without writing a single line of Playwright or Puppeteer.

Request structure:

{
"url": "https://example.com/blog/post",
"formats": ["markdown"]
}

Response structure (abbreviated):

{
"success": true,
"data": {
"markdown": "# Post Title\n\nFull page content as clean markdown...",
"metadata": {
"title": "Post Title",
"description": "Meta description",
"url": "https://example.com/blog/post"
}
}
}

Make.com's HTTP module sends this request, parses the JSON response, and makes every field available as a variable in downstream modules. No custom parsing code needed.

Building the Scenario Step by Step

Step 1: Create a New Scenario

  1. Open Make.com and click Create a new scenario
  2. Click the + button to add the first module
  3. Search for Schedule and select it
  4. Set the interval to Every day at your preferred time (e.g., 9:00 AM)

For testing, switch the trigger to Manually so you can run it without waiting for the schedule.

Step 2: Add an Iterator for Your URL List

If you're monitoring a fixed list of URLs, use the Tools > Set Variable module to define an array, then add an Iterator module to loop through them.

Set Variable configuration:

  • Variable name: urls
  • Variable value (use Make's array notation):
["https://competitor.com/blog", "https://industry-news.com/latest", "https://docs.yourtool.com/changelog"]

Connect an Iterator module after the Set Variable module. Set the Array field to {{1.urls}} (the variable you just defined). The Iterator sends one URL at a time to the next module.

Step 3: Configure the HTTP Module to Call Firecrawl

Add an HTTP > Make a request module after the Iterator. This is where the Firecrawl call happens.

HTTP module configuration:

FieldValue
URLhttps://api.firecrawl.dev/v1/scrape
MethodPOST
HeadersAuthorization: Bearer YOUR_FIRECRAWL_API_KEY
Body typeRaw
Content typeapplication/json
Request contentSee below

Request body:

{
"url": "{{3.value}}",
"formats": ["markdown"]
}

Replace {{3.value}} with the actual module number of your Iterator — Make.com assigns numbers dynamically. The 3.value variable contains the current URL from the loop.

Response configuration:

  • Check Parse response so Make.com automatically deserializes the JSON
  • Set Timeout to 30 seconds (Firecrawl can take 10–20 seconds for JavaScript-heavy pages)

After saving, click Run once and inspect the output. You should see data.markdown populated with the page content.

Step 4: Feed Markdown into an AI Module

Add an OpenAI > Create a Completion module (or Anthropic > Create a Message if you prefer Claude).

OpenAI configuration:

FieldValue
Modelgpt-4o-mini (cost-efficient for summaries)
Messages (role)user
Messages (content)See below

Prompt template:

Summarize the following web page content in exactly 3 sentences. Focus on the main topic, key insight, and any notable update.

Page URL: {{3.value}}
Page Title: {{4.data.metadata.title}}

Content:
{{4.data.markdown}}

Replace module numbers to match your scenario. The {{4.data.markdown}} reference pulls the Firecrawl markdown directly into the AI prompt.

Practical tip: Add a Tools > Set Variable module before the AI call to truncate the markdown if it exceeds your model's context window. Most blog posts are under 3,000 tokens, but documentation pages can run much longer.

{{substring(4.data.markdown; 0; 8000)}}

This caps the markdown at roughly 6,000 tokens before OpenAI processing.

Step 5: Route the AI Output

Add an output module based on where you want the summaries:

  • Google Sheets: Append a row with URL, title, date, and AI summary
  • Slack: Post a message to a monitoring channel
  • Gmail: Send a daily digest email
  • Airtable: Log structured records for trend tracking

For a Google Sheets setup, map these columns:

Sheet ColumnSource
URL{{3.value}}
Title{{4.data.metadata.title}}
Summary{{5.choices[0].message.content}}
Date{{now}}

Practical Use Case: Daily Blog Monitoring Pipeline

Here's the complete scenario logic for monitoring competitor blogs:

Schedule (daily 9 AM)
→ Set Variable (URL array)
→ Iterator (one URL at a time)
→ HTTP Module (Firecrawl /scrape)
→ OpenAI (summarize markdown)
→ Google Sheets (append row)

Real-world configuration adjustments:

  • Dynamic URL source: Replace the hardcoded array with a Google Sheets > Watch Rows module. Maintain a sheet of monitored URLs — Make.com pulls the list fresh on each run.
  • Deduplication: Add a Filter after the HTTP module to skip pages where the metadata title matches a previously logged entry in your output sheet.
  • Error handling: Enable Break/Resume error handling on the HTTP module. If Firecrawl returns a 429 (rate limit), the scenario pauses and retries automatically.

Rate limits to know:

Firecrawl PlanCredits/MonthConcurrent Requests
Free5001
Starter ($16/mo)3,0003
Standard ($83/mo)100,0005

Each /scrape call consumes 1 credit. Monitoring 10 URLs daily costs ~300 credits/month — within the free tier.

For pipelines feeding scraped content into LLMs for RAG pipelines or vector databases, Firecrawl's markdown output is already tokenizer-friendly — no additional HTML stripping needed.

Troubleshooting Common Issues

HTTP 401 Unauthorized: The Authorization header is malformed. Make sure the value is exactly Bearer YOUR_API_KEY — the word "Bearer" followed by a space before the key.

HTTP 402 Payment Required: Your Firecrawl credits are exhausted. Check your usage on the Firecrawl dashboard. The free tier resets monthly.

Timeout errors: Firecrawl can take 15–25 seconds on JavaScript-heavy pages. Increase the Make.com HTTP module timeout to 60 seconds. If pages consistently time out, add the waitFor parameter to the request body:

{
"url": "{{3.value}}",
"formats": ["markdown"],
"waitFor": 3000
}

This instructs Firecrawl to wait an additional 3 seconds after initial page load before capturing the DOM.

Empty markdown output: Some pages return very little content — paywalled articles, login-gated pages, or pages that rely entirely on client-side data fetching after authentication. Add a filter in Make.com after the HTTP module: only proceed if {{length(4.data.markdown)}} > 200 to skip near-empty responses.

Make.com "Data structure mismatch" errors: This happens when the HTTP module receives an unexpected response format. Enable Store all responses in the HTTP module's advanced settings during testing so you can inspect the raw response body and debug the path mapping.

Handling Firecrawl's Response in Make.com

Make.com's HTTP module returns parsed JSON objects when Parse response is enabled. Here's the full response path mapping:

Data You NeedMake.com Variable Path
Page markdown{{moduleN.data.markdown}}
Page title{{moduleN.data.metadata.title}}
Page description{{moduleN.data.metadata.description}}
Source URL{{moduleN.data.metadata.url}}
Status code{{moduleN.status}}

If success is false in the response, add a Router module after the HTTP call with a filter checking {{moduleN.data.success}} = true. Route failed scrapes to a logging module instead of the AI step.

Cost Breakdown for This Pipeline

Running the daily blog monitoring scenario for a month (30 days, 10 URLs/day):

ComponentMonthly Cost
Make.com (Core plan, 10k ops)$9/mo
Firecrawl (Free tier: 500 credits)$0
OpenAI gpt-4o-mini (300 short completions)~$0.05
Total~$9/mo

The Make.com free tier includes 1,000 operations/month, which covers small monitoring pipelines at no cost. Each URL processed uses approximately 5–6 operations (iterator step, HTTP call, AI call, sheet append, error handler).

If you're running larger extraction jobs — thousands of URLs, complex crawls across entire domains — consider pairing Make.com with Apify, which handles distributed scraping at scale and integrates with Make.com via webhooks and the Apify API.

Extending the Pipeline

Once the core scenario works, common extensions include:

Multi-format extraction: Add "formats": ["markdown", "links"] to the Firecrawl request body to also extract all hyperlinks from the page. Useful for tracking which external sources a competitor is citing.

Structured data extraction: Firecrawl's /extract endpoint accepts a JSON schema and uses an internal LLM to populate it from the page. Useful for extracting specific fields (product prices, author names, publication dates) without writing CSS selectors.

MCP integration: Firecrawl supports the Model Context Protocol, meaning AI agents running in Claude Desktop or Cursor can trigger Firecrawl scrapes autonomously. For scenarios where an AI agent determines which pages to scrape, this removes the need for a hardcoded URL list entirely.

Scheduled competitor analysis: Feed summaries into a weekly digest, use a second AI call to identify patterns across multiple summaries (e.g., "What topics appeared in 3 or more competitor posts this week?"), and route the pattern analysis to your content team's Slack channel.

Frequently Asked Questions

Yes. Make.com's HTTP module can send a POST request to Firecrawl's REST API at https://api.firecrawl.dev/v1/scrape. You add your Firecrawl API key as a Bearer token in the Authorization header, pass the target URL in the request body, and Make.com receives the markdown response. No middleware or custom code needed.

The standard approach is a three-module chain: HTTP module (calls Firecrawl /scrape) → variable to truncate markdown if needed → AI module (OpenAI or Claude). Firecrawl handles JavaScript rendering and returns clean markdown. The AI module then receives that markdown as part of the prompt. The full pipeline can be built in Make.com without writing code.

The HTTP > Make a request module in Make.com sends fully customizable HTTP requests — any method (GET, POST, PUT, DELETE), custom headers, JSON or form bodies, and automatic JSON response parsing. It functions as a universal API connector for any REST service that isn't available as a native Make.com integration.

Each /scrape call costs 1 Firecrawl credit. Monitoring 10 URLs daily for 30 days costs 300 credits. Firecrawl's free tier includes 500 credits/month, so small monitoring pipelines run at no cost. The Starter plan at $16/month provides 3,000 credits.

The /scrape endpoint processes a single URL and returns its content as markdown — ideal for known, specific pages. The /crawl endpoint recursively discovers and scrapes all pages within a domain. For a blog monitoring pipeline where you know the exact URLs, /scrape is sufficient and more cost-efficient. Use /crawl when you need to index an entire site without knowing the individual page URLs.


Ready to build? Start your Make.com free account and grab a Firecrawl API key — the free tiers of both tools cover a working monitoring pipeline at zero cost.