Skip to main content
use-apify.com

Automation: guides & tutorials

Automate scraping, APIs, and schedules with Apify. Trigger actors, push datasets downstream, and replace manual copy-paste between your stack and sources.

96 articlesPage 1 of 10

View all tags

Web automation handles the repetitive digital work humans should not: extracting data on a schedule, filling forms, routing results between tools, and firing alerts when a price drops or a review appears. These guides show how to build automations that run 24/7 with no babysitting, using no-code tools or custom code.

Non-coders reach for Make, Zapier, or n8n to trigger scrapers and move data visually, while developers use Playwright and Crawlee for browser-level control. Apify sits at the data layer, feeding any automation tool with fresh structured records. Below you will find tutorials for wiring scrapers into workflows, comparisons of automation platforms, and patterns for reliable scheduled runs.

Related topics

Claude20 min read

Claude Code Headless Mode and Cron: Nightly Jobs on a VPS

· 20 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

claude -p "<prompt>" runs Claude Code without the interactive UI: it does the task, prints the result and exits. Put that command in a shell script, call the script from cron on a small Linux server, and you have an agent that works overnight. The safe version adds four things: a long-lived token in a chmod 600 file, --permission-mode dontAsk with a short allow list, --max-budget-usd and --max-turns caps, and --output-format json so every run leaves a log with its estimated cost. For web pages that block plain fetches, connect the Apify MCP server with an API token.

Most Claude Code guides stop at the interactive terminal. This one is about the jobs you want done before you sit down: a digest of what changed in the tools you depend on, a refreshed dataset, a report waiting in a folder. The example job below was run for real on 2026-10-21, and the flags, costs and failure modes in this post come from that run and from Anthropic's Claude Code documentation.

Claude18 min read

Price Your Airbnb Against Nearby Comps Every Week with Claude

· 18 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

Set up a weekly Claude Cowork scheduled task that runs the Airbnb Scraper API (codepoetry) on Apify for about 50 comparable listings near yours. Claude prices the next four weekends against them, works out each comp's calendar-implied occupancy, pulls the amenities guests praise from their reviews, and writes a one-page pricing memo. A full weekly run costs about $1.27 in Apify usage on the free plan, or about $0.79 on Starter.

"How should I price my Airbnb?" has one honest answer: the same way the listings guests compare you with are priced, on the same dates. That comp set changes every week. New listings open, neighbours drop their rates for a quiet weekend, and a festival fills the calendar three streets over. This post sets up a Monday-morning memo that tracks it for you, with no code.

Claude16 min read

Claude for Recruiting: Build a Candidate Shortlist from Job Boards and Team Pages

· 16 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

Paste a role spec into Claude Cowork. Claude uses Apify scrapers to pull job ads for the same role in your market, turns the hiring companies into a target list, reads those companies' public team pages, and scores the people it finds against your must-haves. Every score cites the line it came from. You review the list and decide who to contact; Claude only drafts outreach for the people you approve. Scraping for one role costs well under a dollar. The legal limits (GDPR, LinkedIn's and GitHub's terms) shape the workflow more than the tooling does, so they get their own section.

Sourcing tools sell you a database. This workflow builds a small, fresh one per role from pages that companies chose to publish, and it keeps a person in charge of every decision about a human being. That second part is not decoration: in the EU, the same scoring step that saves you an afternoon is the part regulators care most about.

Claude17 min read

Claude Code Web Research Setup: CLAUDE.md, MCP and Hooks (2026)

· 17 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

Four files turn a folder into a Claude Code research workspace: a CLAUDE.md with sourcing rules, a project-scope .mcp.json that connects the Apify MCP server with only three tools (search-actors, fetch-actor-details, call-actor), a .claude/settings.json that wires a PreToolUse hook to refuse any Actor run without a spend cap of $1 or less, and a /research skill that runs the whole loop. Every file is below, ready to copy.

Most Claude Code setup guides are about writing code. Mine is mostly about reading the web: pulling search results, pages Claude's built-in fetch can't render, and datasets I want to keep. That job has two failure modes code work does not have. Claude can state things it never sourced, and a scraper can quietly spend money.

This setup addresses both. It is a folder you can recreate in five minutes, and it works the same for a solo researcher as for a team that commits it to a repo.

Claude15 min read

Claude Job Search: A Daily Digest of New Jobs Ranked Against Your CV

· 15 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

Create a Claude Cowork scheduled task that runs every morning. It calls two Apify job scrapers (LinkedIn and Indeed) for postings from the last 24 hours, scores each one against the CV you pasted into the task, and gives you the top 5 with tailoring notes. The notes can only reuse lines from your CV, so Claude cannot invent experience for you. At the job volumes below, the scraping stays inside Apify's free $5 monthly credit.

Job alerts from the boards themselves match on keywords, so you get 40 emails a week and skim them all. The job that matters is the reverse: read every new posting, compare it to what you have actually done, and surface the few worth an application today. That is reading work, which is what Claude is good at, provided something fetches the postings for it.

Claude16 min read

Reddit Brand Monitoring in Claude Cowork: Threads Worth Answering

· 16 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

Connect Apify to Claude Cowork, then schedule a daily task that searches Reddit for your brand and competitors with the Reddit Scraper Lite Actor. Claude throws out the false matches, tags the sentiment of the real mentions, and lists the threads where a reply from you would help. In our test, only 1 of 8 Reddit posts that matched a brand search was a real conversation about the product, so the filtering step is the actual job. A daily run of 30 items costs about $0.16 at the Free-plan list price.

Reddit is where people ask "is X worth it?", complain about a pricing change, and recommend tools to strangers. Those threads also get read by AI assistants. In Ahrefs' September 2026 data, reddit.com was the most-cited domain in ChatGPT answers for broad US queries, with 16.8% of citations (Ahrefs, updated 2 September 2026). That share moves around: Semrush saw ChatGPT cite Reddit in close to 60% of responses in early August 2025 and around 10% by mid-September (Semrush, 10 November 2025). The ranking is consistent even when the percentage is not, so an unanswered complaint thread about your product can end up shaping AI answers about you as well.

Claude16 min read

Respond to Google Reviews with Claude Across 10 Locations

· 16 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

Set up a Claude Cowork scheduled task that runs once a week. It calls the Google Maps Reviews Scraper on Apify for every location's reviews from the last 8 days, skips the ones you already answered, and drafts a reply for each new one in your brand voice. Anything rated 3★ or lower is put at the top, risky reviews are escalated with no draft, and sudden bursts of reviews are flagged for a human to judge. Claude drafts; you post. Google only publishes a reply when the business posts it from its Business Profile or through Google's own API. For 10 locations the scraping costs well under $1 a week.

Answering every review across ten locations is not hard work. It is tedious work, so it slips: the 5★ reviews get a "thanks!", the 2★ review from three weeks ago never gets an answer, and nobody notices the location whose rating is quietly sliding. A weekly task fixes the slipping part. It reads every new review, writes the first draft, and hands you a single list to approve.

Claude5 min read

Claude Cowork Scheduled Tasks: Web-Data Digests on Autopilot (2026)

· 5 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

Claude Cowork can run a task on a schedule (hourly, daily, weekly, or weekdays) and it runs even when your computer is asleep. Pair that with an Apify scraper and you get a recurring web-data digest, a morning price check or a weekly review report, delivered without you doing anything.

Most scraping guides stop at the one-off run. The real leverage is recurring data: the prices, reviews, and listings that change while you are not watching. Cowork's scheduled tasks turn a single scrape into a standing job.

Claude6 min read

Build a Claude Skill for Web Scraping (Calls Apify Actors) (2026)

· 6 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

A Claude Skill is a SKILL.md folder that packages instructions Claude loads automatically when relevant. Build one that tells Claude to run Apify Actors for scraping, and every future "get me this web data" request just works. Or install the official Apify Claude Code plugin, which ships five ready-made skills.

Agent Skills are modular capabilities that extend Claude: each Skill packages instructions, metadata, and optional resources that Claude uses automatically when they are relevant to your request. Instead of re-explaining how you like to scrape every time, you encode it once, and Claude reaches for it on its own.

This guide builds a scraping skill that calls Apify, and points you at the official plugin if you would rather not start from scratch.

Claude5 min read

Claude Cowork vs Claude Code: Which for Web Data and Automation? (2026)

· 5 min read
Yassine El Haddad
Software Developer & Automation Specialist
Quick Answer

Claude Code and Claude Cowork run on the same engine, but they are built for different people. Use Cowork to delegate knowledge work (research, analysis, reports) with no terminal, and Code to build software in your project. For web data, both can call Apify Actors: pick Cowork to run scrapers and get a spreadsheet, Code to build and automate a data pipeline.

Anthropic ships both Claude Code and Claude Cowork on the same agentic architecture, which confuses a lot of people into thinking they overlap. They do not. The difference is who each is for and where it runs.

Guides on this site

Frequently asked questions

Frequently Asked Questions

Automation handles any repetitive digital task: filling and submitting web forms, extracting data on a schedule, routing results between tools, sending alerts when a price drops or a review appears, syncing records across CRMs and databases, and triggering follow-up workflows. Teams use automation to eliminate hours of daily manual work — once set up, it runs 24/7 without intervention.

For non-coders: Make.com, Zapier, or n8n connect apps visually and can trigger Apify scrapers without writing code. For developers: Playwright or Puppeteer automate browsers; Crawlee adds queuing and cloud deployment. Apify acts as the data extraction layer that feeds any automation tool. The right stack depends on whether you need browser interaction, data transformation, or just API calls between services.

Simple automations — schedule a scraper, send results to a Google Sheet, trigger a Slack alert — take under an hour with Apify and Make.com. Custom browser automations that interact with complex sites typically take a few days to build and test. Deployment services on this site deliver ready-to-run automation stacks in 1–3 business days with full documentation, so you skip the setup phase entirely.

Scraping extracts data; automation performs actions. In practice they overlap — automated login precedes many extractions, and scrapers use automation to click through paginated results. The distinction matters for compliance: read-only data extraction and write-action form submission carry different legal risk profiles and need different session management strategies.