Apify AI Agent & Vector Database Integrations
Apify connects to AI agents three ways: (1) the Apify MCP server (apify/actors-mcp-server, hosted at https://mcp.apify.com) exposes Store Actors as tools to any MCP client; (2) framework integrations for LangChain, LlamaIndex, CrewAI, LangGraph, the OpenAI Agents SDK, the Vercel AI SDK, Google ADK, and more; and (3) vector-database integrations (Pinecone, Qdrant, Milvus) that push scraped data into a RAG index. Apify also supports agentic payments so agents can run Actors without a human account (integrations docs).
How does Apify integrate with AI agents? Apify turns its 30,000+ Store Actors into callable agent tools three ways: the Apify MCP server (https://mcp.apify.com) for any MCP client, native packages for frameworks like LangChain, CrewAI, and LlamaIndex, and direct REST API or client-library calls. Agents use these tools to scrape live web data, then optionally push it to a vector database for RAG.
If you are building an AI agent that needs live web data, Apify gives it real-time scraping and automation tools. This guide maps the integration options so you can pick the right one. For the no-code MCP setup walkthrough, see Use Apify MCP servers.
Three ways to connect an agent
| Path | What it is | Use when |
|---|---|---|
| MCP server | A Model Context Protocol server that exposes Actors as callable tools | Your client speaks MCP (Claude Desktop, Cursor, VS Code, ChatGPT, and others) |
| Framework integration | A native package or tool wrapper for an agent framework | You build agents in code with LangChain, CrewAI, LlamaIndex, etc. |
| Client libraries / REST API | The JavaScript and Python clients, or the REST API | You want full control or your framework has no native integration |
The Apify MCP server
The Apify MCP server is the broadest path: one connection gives an agent access to thousands of Store Actors as tools, with dynamic tool discovery.
- Hosted endpoint:
https://mcp.apify.com(Streamable HTTP transport; the older SSE transport was retired in 2026). - Local package:
@apify/actors-mcp-servervianpx. - Auth: OAuth on first connect, or an Apify API token.
- One-click installs: available for Claude Desktop (connector directory) and Cursor / VS Code.
Full setup, including Claude Desktop, is in Use Apify MCP servers.
The server ships built-in tools your agent calls directly: search-actors to find an Actor in the Store, fetch-actor-details to read its input schema, and add-actor to load a new Actor as a tool on demand (dynamic tool discovery). This means an agent can discover and wire up a scraper at runtime rather than being limited to a fixed tool list. See Best MCP server Actors for the Actors that work best as agent tools.
Actors as live-web tools for agents
The most common agent task is fetching current web data. Two Actors are built for exactly this:
- RAG Web Browser queries Google Search, scrapes the top results with a full browser, and returns clean Markdown ready for an LLM prompt. It runs in standby mode as an HTTP endpoint, so an agent can hit it for real-time search-and-read in one call. It is also a built-in tool on the Apify MCP server.
- Website Content Crawler crawls a whole site or doc set and returns Markdown, the standard first step before building a RAG index.
Any of the 30,000+ Store Actors can act as a tool. The pattern is the same: the agent calls the Actor (via MCP, a framework wrapper, or the API), the Actor returns a dataset, and the agent reasons over the results. Browse Best AI & RAG data Actors for ready-made extractors.
Agent framework integrations
Apify maintains native integrations for the major agent and LLM frameworks. Each lets your agent call Actors for web data and automation.
| Framework | What the integration does | Official docs |
|---|---|---|
| LangChain | langchain-apify package: ApifyWrapper, ApifyDatasetLoader, ApifyActorsTool | docs · our guide |
| LlamaIndex | Load Actor results into a LlamaIndex vector store or use Actors as tools | docs |
| CrewAI | Equip multi-agent crews with Apify tools for scraping and extraction | docs |
| LangGraph | Build stateful, multi-step agent workflows that search and extract web data | docs |
| OpenAI Agents SDK | Use Actors as tools via the Apify MCP server inside the OpenAI Agents SDK | docs |
| OpenAI Assistants | Provide real-time web data and store results in an OpenAI Vector Store | docs |
| Vercel AI SDK | Add Actors as tools for TypeScript agents | docs |
| Google ADK | Use Actors as tools for agents built with the Agent Development Kit | docs |
| Agno / Mastra / Haystack | Tool and data-source integrations for these agent frameworks | Agno · Mastra · Haystack |
| Flowise / Langflow | Use Actors as data sources in low-code visual agent builders | Flowise · Langflow |
| Amazon Bedrock | Supply Bedrock Agents with web data via the RAG Web Browser Actor | docs |
Most of these call Actors through the MCP server or the API client under the hood, so the pay-per-event cost of each Actor run is what your agent pays.
Vector database integrations (RAG)
For retrieval-augmented generation, the common pattern is: crawl with Website Content Crawler (or the RAG Web Browser), then push the dataset into a vector database. Apify has native export integrations:
| Vector DB | Integration | Official docs |
|---|---|---|
| Pinecone | Export Actor results into a Pinecone index for similarity search | docs |
| Qdrant | Export dataset items into a Qdrant collection | docs |
| Milvus | Export results into a Milvus collection for semantic search | docs |
These run as integrations on your Actor or task, so embeddings flow to the index after each successful run. For the end-to-end pattern, see Data for AI & RAG.
Agentic payments (autonomous agents)
Apify supports two protocols that let an AI agent discover and pay for Actor runs without an Apify user account:
- x402: agents pay per Actor run using USDC on the Base blockchain (docs).
- Skyfire: agents autonomously discover and run Actors through Skyfire's agent-payment network (docs).
These are early but signal where agent-to-tool commerce is heading: an agent can find a scraper, run it, and settle payment programmatically.
Which path should I pick?
- No code / fastest: the MCP server. One connection, thousands of tools, works with Claude, Cursor, ChatGPT, and more.
- Building in Python: LangChain, LlamaIndex, or CrewAI; pair with Pinecone/Qdrant/Milvus for RAG.
- Building in TypeScript: the Vercel AI SDK or the OpenAI Agents SDK over MCP.
- Fully autonomous / machine-to-machine: x402 or Skyfire agentic payments.
Start on the Apify Free plan ($5 monthly credits, no card), connect the MCP server, and let your agent call a Store Actor.
The Apify MCP server. Connect your MCP-compatible client (Claude Desktop, Cursor, VS Code, ChatGPT) to https://mcp.apify.com or run @apify/actors-mcp-server locally, and your agent can call thousands of Store Actors as tools without custom code.
Yes. LangChain has the official langchain-apify package (ApifyWrapper, ApifyDatasetLoader, ApifyActorsTool). LlamaIndex can load Actor results into a vector store or use Actors as tools. See the linked official docs for each.
Crawl with Website Content Crawler or the RAG Web Browser, then use Apify's Pinecone, Qdrant, or Milvus integration on your Actor or task to push dataset items into the index after each run. The Data for AI and RAG guide walks through the full pipeline.
Whatever the Actor's pricing model charges. Most Store scrapers are pay per event, so the agent pays a fixed price per result or page. The free plan's $5 monthly credit covers small test runs. See the pay-per-event guide.
Yes, through agentic payments. The x402 protocol lets an agent pay per run with USDC on the Base blockchain, and Skyfire lets agents discover and run Actors through its payment network, both without a human-created Apify account.
The RAG Web Browser Actor is built for this. It queries Google Search, scrapes the top pages with a real browser, and returns clean Markdown the agent can use immediately. It runs as an HTTP standby endpoint and is also a built-in tool on the Apify MCP server, so an agent can search and read the live web in a single call.
LangChain, LlamaIndex, CrewAI, LangGraph, the OpenAI Agents SDK, OpenAI Assistants, the Vercel AI SDK, Google ADK, Agno, Mastra, Haystack, Flowise, Langflow, and Amazon Bedrock, among others. Most route through the MCP server or the API client.
Sources