Apify vs. Crawl4AI: the real cost of self-hosting
Crawl4AI is a free Apache-2.0 Python library you install and run on your own machine. Apify is a managed platform whose cheapest paid plan is $19/month. Crawl4AI is the better answer when one server can hold the whole job and you are willing to patch it. Apify is the better answer when you need concurrency, proxies, scheduling and storage that somebody else keeps running. The feature lists barely differ. The maintenance bill does.
Crawl4AI shipped v0.9.3 on 2026-08-31, needs Python 3.10 or newer, and drives Chromium through Playwright (GitHub API and PyPI, checked 2026-09-09). Apify hosts a Store of 70,000+ ready-to-run Actors and bills compute by the gigabyte-hour. This page prices both, and it is blunt about the parts nobody can price for you.
:::note What changed on this page, 2026-09-09
Two Apify numbers anchor this comparison: Starter is $19/month ($17 billed annually), verified at apify.com/pricing on 2026-09-09, and the Store's own published figure is 70,000+ Actors. The cost comparison below is built on those.
:::
What is the actual difference between Apify and Crawl4AI?
| Crawl4AI | Apify | |
|---|---|---|
| What it is | Python library you import | Cloud platform you call |
| License | Apache-2.0 | Proprietary SaaS, free tier available |
| Current release | v0.9.3, published 2026-08-31 | Continuously deployed |
| Runtime | Python 3.10+, Playwright/Chromium | Docker Actors, JS and Python SDKs |
| Where it runs | Your laptop, VPS, or Docker host | Apify's cloud |
| Pre-built scrapers | None; you write the extraction | 70,000+ Store Actors |
| Headline output | LLM-ready Markdown | Structured dataset records; Markdown from specific Actors |
| Proxies | You buy and wire them in | Sold per GB on the platform, or bring your own |
| Scheduling | Your cron, your dispatcher | Built in, plus webhooks and API triggers |
| Storage | Your database or filesystem | Datasets and key-value stores |
| Concurrency ceiling | Your RAM | Plan limit (see below) |
| Who patches it | You | Apify |
| Licence cost | $0 | $0-999/month by plan |
Sources: github.com/unclecode/crawl4ai and docs.crawl4ai.com for Crawl4AI; apify.com/pricing and apify.com/store for Apify. All fetched 2026-09-09.
What does Apify actually charge?
Verified on apify.com/pricing, 2026-09-09:
| Plan | Monthly | Annual, per month | Included usage | Compute price | Max RAM | Max concurrent runs |
|---|---|---|---|---|---|---|
| Free | $0 | $0 | $5 | $0.20 / CU | 16 GB | 5 |
| Starter | $19/mo subject to change · verified 2026-09-09 | $17 | $19 | $0.20 / CU | 64 GB | 32 |
| Scale | $199 | $179 | $199 | $0.16 / CU | 256 GB | 128 |
| Business | $999 | $899 | $999 | $0.13 / CU | 512 GB | 256 |
The monthly fee is not an access fee. It is prepaid usage credit that your consumption is billed against, and Apify states that unused credit does not roll over. You pay more only when the month's usage exceeds the included amount.
One compute unit is one gigabyte of RAM running for one hour (Apify docs, usage and resources, fetched 2026-09-09). The same docs page lists three things billed on top of compute: data transfer, storage operations, and residential or SERP proxy traffic. Residential proxy is $8/GB on the Free and Starter plans.
Why this page will not hand you a total. A cost-per-1,000-pages table would be invented. Compute depends on how much RAM you allocate and how long each page takes, and both vary by target site. The honest method takes ten minutes: run 100 of your real URLs, read the compute units off the run detail page, multiply by 10 for a per-1,000 figure, then multiply by $0.20. Do the same measurement on your own hardware before you assume self-hosting is cheaper.
What does Crawl4AI cost once it is in production?
The library is $0 forever. The production line items are yours:
- A host. Playwright plus Chromium is not a 512 MB workload. Budget a server that can hold your browser concurrency.
- Proxies. Crawl4AI supports proxies with authentication, but it does not sell them. On any site that blocks datacenter IPs you are buying residential bandwidth from a third party, which is the same line item Apify charges $8/GB for.
- Patching. See the next section. This is the cost people leave out.
- Parser upkeep. Crawl4AI returns clean Markdown or structured HTML. Deciding which fields matter, and repairing that when a site changes, stays your job.
How many pages can you run at once? On your own box the ceiling is RAM. On Apify the ceiling is your plan, and here Apify's two tier-1 pages disagree about the Free plan. apify.com/pricing lists 5 max concurrent runs for Free; docs.apify.com/platform/limits lists 25 for the same plan. Both fetched 2026-09-09. We have not picked a side and neither should you: design for 5, because the lower of two conflicting official numbers is the one that will not break your pipeline. The paid tiers agree across both pages at 32, 128 and 256.
What does maintaining Crawl4AI actually cost you?
This is the part the feature tables never show, and it is not hypothetical. Read Crawl4AI's own release notes:
| Release | Date | What it was |
|---|---|---|
| v0.8.7 | Security hardening | Fixed critical Docker API vulnerabilities: sandbox-escape RCE, hardcoded JWT secret, SSRF, arbitrary file write, monitor auth bypass, stored XSS. The notes say "If you self-host the Docker API, upgrade immediately." |
| v0.9.0 | Secure-by-default rework | Auth on by default; the server binds loopback unless given a token; the request body treated as an untrusted boundary. |
| v0.9.3 | 2026-08-31 | Five coordinated-disclosure advisories: arbitrary file write, SSRF, and denial of service in the PDF path, plus two XSS issues, one in the PDF path and one in the Docker Playground. Thirty-three bug fixes alongside. |
Source: the Crawl4AI README release history, fetched 2026-09-09.
Three consecutive releases carrying security fixes for a self-hosted API server is not a knock on the project. It is a healthy disclosure process, and the maintainers ship fixes fast. It is a statement about who is on call. If you expose Crawl4AI's Docker server, you own that upgrade window, including the one that lands while you are on holiday. If that sentence made you wince, price your own hours into the comparison before you call the library free.
The mirror image is true for Apify: you are trusting somebody else's patch cadence and you cannot audit it. That is the trade, stated plainly in both directions.
Is Crawlee the better open-source option?
Often, yes, and it is worth saying on a page selling the managed platform.
Crawlee is Apify's own crawling library, Apache-2.0 and free to run anywhere. The JavaScript package is at 3.18.1 (published 2026-08-12) and Crawlee for Python is at 1.10.0 (published 2026-08-31), both verified on the npm and PyPI registries on 2026-09-09. It handles request queues, retries, session rotation, proxy configuration and autoscaling: the plumbing you would otherwise hand-roll around Crawl4AI.
Pick between the two on what you are actually building:
| You are building | Better library |
|---|---|
| Markdown extraction from arbitrary URLs for RAG | Crawl4AI |
| A crawler with queues, retries and sessions across thousands of URLs | Crawlee |
| Something you may later want to run on managed infrastructure without a rewrite | Crawlee |
| A Python-only stack with LLM-driven extraction strategies built in | Crawl4AI |
Crawlee's structural advantage is the exit. The same Crawlee code runs on your server and, wrapped as an Actor, on Apify's. The managed-versus-self-hosted decision stops being a rewrite and becomes a deployment target. Our Crawlee for Python tutorial walks through that path. Crawl4AI has no equivalent migration route; moving that workload to Apify means porting it.
Which is better for RAG and LLM pipelines?
For one machine feeding one vector database, Crawl4AI is the simpler and cheaper answer, and this site has no interest in pretending otherwise. Markdown is its headline output, the install is two commands, and there is no billing to reason about.
Apify's equivalents earn their keep at a different point. Website Content Crawler strips navigation and boilerplate and returns clean text or Markdown; the Apify API reported over 155,000 users and over 42 million runs on 2026-09-09. RAG Web Browser searches and returns page content in one call, at over 169,000 users on the same date. Both are billed pay-per-usage: Apify's pricing page defines that as "the developer charges nothing on top, and you only pay for the platform usage consumed by the Actor's runs." Neither carries a rental fee.
Both are covered in more depth in our guides to crawling site content into Markdown and the best Apify Actors for AI data.
The one capability with no self-hosted equivalent is agent access. Apify runs a hosted Model Context Protocol endpoint at https://mcp.apify.com that lets an AI agent discover and run Store Actors and read the results back (Apify MCP docs, fetched 2026-09-09). With Crawl4AI you would build and host that surface yourself.
Which should you pick?
| Your situation | Pick | Why |
|---|---|---|
| Learning, prototyping, or a one-off extraction | Crawl4AI | Nothing to sign up for, nothing to bill |
| Feeding one RAG index from a handful of domains | Crawl4AI | Markdown out of the box, single machine |
| You have an on-call rotation and like owning the stack | Crawl4AI or Crawlee | The patching cost is one you already pay |
| Nobody on the team wants to own a CVE window | Apify | Patching is the platform's problem |
| You need a scraper for Amazon, Maps, LinkedIn, TikTok | Apify | The Store already has one |
| Scheduled runs with storage and exports, no glue code | Apify | Cron, datasets and webhooks are built in |
| Concurrency beyond what one server holds | Apify | 32 concurrent runs on Starter, 128 on Scale |
| An AI agent needs to call the scraper as a tool | Apify | Hosted MCP endpoint |
| You want to keep the option of switching later | Crawlee | Same code local or on-platform |
The obvious objection: "$19/month for something I can run for free." Correct, and if your workload fits on one box and you are comfortable patching a Playwright-based Docker server, run Crawl4AI. Apify's $19 buys 32 concurrent runs, managed proxies, and a patch cadence that is not your problem. Below roughly one engineer-hour of monthly maintenance, self-hosting wins on cost. Above it, it does not.
Start on Apify's free plan ($5 of monthly usage credit, no card at signup) and measure your own compute units before committing to a paid tier. Or install Crawl4AI and measure the same job on your own hardware. Both measurements take an afternoon and they settle this argument better than any comparison table.
The library is free and open-source under Apache-2.0, and it always will be. Production is not free: you pay for the server, for proxies on sites that block datacenter IPs, and for the hours spent applying security updates. Apify's free plan includes $5 of monthly usage credit, and its cheapest paid plan is $19/month, or $17 when billed annually (apify.com/pricing, verified 2026-09-09).
Apify Starter is $19/month, which includes $19 of usage credit that does not roll over, and compute is billed at $0.20 per compute unit (one gigabyte of RAM for one hour). Crawl4AI's software cost is zero and its infrastructure cost depends entirely on your host and proxy spend. Measure both on 100 of your real URLs rather than trusting any generic table, including this one.
No. Crawl4AI is a library for building crawlers; the extraction logic is yours to write and maintain. Apify's Store publishes 70,000+ ready-to-run Actors, which is the practical reason teams stop hand-writing parsers for heavily defended sites.
Apify's two official pages disagree, and both were checked on 2026-09-09. The pricing page lists 5 concurrent runs on the Free plan; the platform limits page lists 25. Plan for 5 until Apify reconciles them. The paid tiers are consistent across both pages: 32 on Starter, 128 on Scale, 256 on Business.
If you want queues, retries, session rotation and autoscaling, Crawlee gives you more of that plumbing than Crawl4AI does, under the same Apache-2.0 licence. Its other advantage is portability: the same Crawlee code runs on your own server or wrapped as an Apify Actor, so migrating later is a deployment change rather than a rewrite. For pure Markdown extraction from arbitrary URLs, Crawl4AI is the more direct tool.
Three of its recent releases carried security work on the self-hosted Docker API server. v0.8.7 fixed critical vulnerabilities including a sandbox-escape RCE, a hardcoded JWT secret and SSRF, with the note to upgrade immediately. v0.9.0 made the Docker server secure by default. v0.9.3, published 2026-08-31, closed five coordinated-disclosure advisories. The project handles disclosure well; the point is that applying those upgrades on time is your job, not a vendor's.
Not without building it. Apify runs a hosted MCP endpoint at mcp.apify.com that lets agents discover and run Store Actors and read results back. With Crawl4AI you would expose and secure that interface yourself, which is a second service to patch alongside the crawler.
Yes, but it is a port rather than a lift. You would rebuild the crawl as an Actor, typically on Crawlee, or replace it with an existing Store Actor for that site. If keeping a clean migration path matters to you, starting on Crawlee rather than Crawl4AI removes the rewrite entirely.