Skip to main content
use-apify.com

Agents: guides & tutorials

Build autonomous AI agents that browse, extract, and act on web data. Tools, frameworks, and Apify integrations for reliable agent-based automation.

4 articles

View all tags

AI agents browse, extract, and act on web data without brittle, hand-coded selectors. These guides cover the frameworks, tools, and Apify integrations that make agent-based automation reliable rather than flaky.

Dependable agents need guardrails, retries, and well-defined tools, not just a clever prompt. Apify actors give agents a stable scraping capability to call. Below you will find tutorials on orchestrating agents for real extraction tasks.

Related topics

Agents6 min read

browser-use: Architecting AI-Powered Web Agents (2026)

· 6 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

browser-use is an open-source Python library that gives an LLM control of a real browser (via Playwright). It runs a perceive–act loop: the page DOM is pruned and tagged, the model chooses actions like click or type, and the loop repeats until the task finishes. It shines when layouts change often and fixed selectors break; it costs more in tokens and time than traditional scrapers, and it is a poor fit for hard WAFs or huge deterministic crawls. For production, run it in containers on Apify with proxy rotation.

Traditional automation (Playwright or Puppeteer) depends on stable selectors. If a team hardcodes .submit-btn and the site renames classes, the job fails.

browser-use inverts that: you describe the goal in natural language, the library feeds a sanitized view of the page to an LLM, and the model plans clicks, typing, and extractions through Playwright.

This guide covers the architecture, where it breaks in production, and how to pair it with Apify Actors when you need cloud browsers and proxies.

Agents5 min read

Scrapling + OpenClaw: Connect Adaptive Scraping to an AI Agent

· 5 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

Scrapling gives you adaptive DOM parsing (auto-healing selectors, stealth fetchers). OpenClaw is a local gateway that routes chat channels (e.g. Slack, Discord) to an LLM. Apify runs your Scrapling code in a container with proxies and scaling, so your agent calls a stable HTTP API instead of scraping from your home IP. The usual pattern: Scrapling in an Actor → OpenClaw skill calls run-sync-get-dataset-items → you reply with trimmed markdown to the user.

Scrapling is a Python framework that uses structural similarity to keep selectors working when markup changes. OpenClaw is a local daemon that connects messaging channels to autonomous LLM workflows. Used together, you can say “check competitor prices” in chat and have the agent trigger a cloud scraper that handles WAFs and headless rendering.

This tutorial wires those pieces: Scrapling extraction inside an Apify Actor, triggered from an OpenClaw Node skill via the Apify REST API.

Agents6 min read

OpenClaw: Build a Local, Multi-Channel AI Agent in 2026

· 6 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

OpenClaw is an open-source personal AI assistant you run yourself: a Gateway on your machine routes WhatsApp, Telegram, Discord, and other channels to an LLM (e.g. Ollama locally or OpenAI / Anthropic in the cloud). It is not the model—it is the plumbing (sessions, pairing, tools, optional browser automation). NemoClaw (NVIDIA, 2026) is an enterprise-oriented secure runtime layer for running OpenClaw-class agents inside a hardened, policy-controlled environment—think governance and sandboxing on top of the same assistant idea.

Most people use hosted assistants (ChatGPT, Claude, Gemini). That is convenient, but sensitive threads, internal repos, and private DMs pass through someone else’s servers. For engineers and privacy-conscious teams, OpenClaw (successor lineage to projects sometimes referred to as Moltbot / Clawdbot) flips the model: you host the Gateway, you pick the model, and you decide which tools (files, shell, HTTP, browser, custom skills) the agent may use.

As of March 2026, OpenClaw is among the largest open-source AI assistant projects on GitHub (on the order of hundreds of thousands of stars—exact counts move quickly; check the repo for the live number). This guide explains what it is, how to run it, where it breaks down operationally, and how to pair it with Apify when local scraping is too fragile for production sites.

Agents5 min read

What Google's WebMCP Means for the Future of Web Scraping

· 5 min read
Yassine El Haddad
Software Developer & Automation Specialist

In February 2026, Google unveiled an early technical preview of WebMCP (Web Model Context Protocol), a proposed web standard incubated by the W3C Web Machine Learning Community Group in collaboration with Microsoft. Introduced in Chrome 146 Canary, its architecture proposes a radical shift: rather than forcing AI agents to parse opaque DOM structures via CSS selectors, websites explicitly declare deterministic API endpoints that local agents can execute.

If adopted, WebMCP fundamentally alters the economics of web data extraction. This technical analysis explores how the protocol operates, its architectural failure modes, and what data engineers must build to maintain extraction pipelines through to 2030.

Guides on this site

Frequently asked questions

Frequently Asked Questions

An AI agent can autonomously research companies, compile competitor intelligence, find and qualify leads from public directories, monitor news and surface relevant stories, or navigate multi-step websites that require login or pagination — all without step-by-step instructions. You describe the goal; the agent plans and executes. Results arrive as structured data, not as tabs left open in a browser.

Expose a web scraping tool to the agent — either the Apify MCP server (which lets Claude and other assistants call actors directly) or a custom tool that wraps the Apify REST API. Define allowed domains, output schema, and maximum iterations to keep costs and scope predictable. Start with a narrow task before letting the agent plan freely.

LangGraph for stateful multi-step workflows, CrewAI for multi-agent collaboration, and LangChain for simpler pipelines are the most adopted in 2025. Apify integrates with all three. For non-developers, Claude with the Apify MCP server gives agent-like capabilities in a chat interface without writing framework code.

Cost scales with tokens used per task step, number of iterations, and browser time for headless scraping. Agents that loop autonomously can spend 5–20x more than a single deterministic scraper on the same data. Cap iteration limits, cache HTML between steps, and batch LLM calls where possible. Many production teams use agents only for the ambiguous long-tail and keep bulk extraction deterministic.