Apify JavaScript SDK and Crawlee: build Actors from scratch

The Apify SDK for JavaScript ties your scraper to Apify storage, runs, and platform APIs. Crawlee (the crawling library Apify maintains) handles browsers, queues, sessions, and routing. Together they are the standard way to ship Node.js Actors that run locally and in the cloud.

Official references: Apify SDK for JavaScript and Crawlee docs.

Quick Answer

The Apify JavaScript SDK lets you build and run Actors locally and in the cloud. Install with npm, initialize with Actor.init(), use datasets for output, and deploy with apify push.

At a glance:

npm install -g apify-cli: CLI for create/run/push.
apify create: starter template with Crawlee + Playwright.
await Actor.init(): connect to Apify storage when running on platform (and local emulated storage when running via CLI).
Actor.pushData(): append rows to the default dataset.
apify run executes locally; apify push uploads to your Apify account.

Prerequisites

Node.js 18+ (LTS recommended) and npm.
An Apify account (sign up free).
Optional: Apify documentation open in another tab for API details.

Step 1: Install the Apify CLI

npm install -g apify-cli

Verify with apify --version.

Step 2: Create a project

From a parent directory (for example your projects folder):

apify create my-js-scraper
cd my-js-scraper

When prompted, choose a template such as Hello world in JavaScript or a Crawlee + Playwright starter (recommended for real sites). The CLI generates src/main.js (or main.ts), Dockerfile, and package.json wired to apify and crawlee.

Step 3: Minimal Actor skeleton

A tiny Actor initializes the SDK, writes one record, and exits:

import { Actor } from "apify";

await Actor.init();
await Actor.pushData({ message: "Hello from Apify SDK", url: "https://apify.com" });
await Actor.exit();

On the Apify platform, pushData lands in the run’s default dataset. Locally, apify run writes under ./storage/datasets/default.

Step 4: Add Crawlee + Playwright crawling

Replace src/main.js with a crawler that visits pages, extracts titles, and enqueues links (cap breadth with maxRequestsPerCrawl while learning):

import { Actor } from "apify";
import { PlaywrightCrawler } from "crawlee";

await Actor.init();

const crawler = new PlaywrightCrawler({
    maxRequestsPerCrawl: 50,
    async requestHandler({ request, page, enqueueLinks }) {
        const title = await page.title();
        console.log(`Title of ${request.loadedUrl} is '${title}'`);
        await Actor.pushData({ title, url: request.loadedUrl });
        await enqueueLinks();
    },
});

await crawler.run(["https://crawlee.dev"]);
await Actor.exit();

Why PlaywrightCrawler? It launches a real browser, executes on-page JavaScript, and fits SPAs and anti-bot scenarios better than raw HTTP for many targets.

Step 5: Run locally

cd my-js-scraper
apify login
apify run

Inspect ./storage/datasets/default/*.json for pushed items. Fix errors in the log before you push to production.

Step 6: Deploy with `apify push`

apify push

This uploads your project and creates or updates an Actor in your Apify account. Open the Apify Console, select the Actor, and click Start to run in the cloud. Results appear under Storage → Datasets.

TypeScript

For production Actors, prefer the TypeScript template: stronger types for Dataset, KeyValueStore, and Crawlee handlers. The flow is the same: Actor.init(), crawl, Actor.pushData(), Actor.exit().

Core concepts map

Concept	Role
`Actor.init()`	Bootstraps SDK; wires dataset/KV store when running on Apify or emulated locally
`Actor.pushData(item)`	Appends one object to the default dataset
`Dataset.open()`	Optional named datasets for multi-table exports
`KeyValueStore`	Screenshots, HTML snapshots, arbitrary blobs
`PlaywrightCrawler`	Browser-based crawling with Crawlee session and queue management

Next steps

Scraping dynamic websites: forms, logins, infinite scroll.
Processing scraped data: normalize and ship to warehouses.
Build an Actor: metadata, inputs, publishing to the Store.
Apify MCP: trigger runs from AI assistants.

Browse the full Apify JS docs →

Frequently Asked Questions

Crawlee is an open-source crawling library (queues, HTTP, Cheerio, Playwright). The Apify SDK adds platform integration: Actor lifecycle, dataset and key-value storage APIs, secrets, and deployment. In Actors you usually import both.

From the project root run apify push after apify login. You can also connect GitHub for automatic builds. The Console then lets you run, schedule, and monitor the Actor.

CLI templates add Playwright as a dependency. If you start from scratch, add playwright and crawlee to package.json and follow Crawlee’s browser install notes (browsers download on postinstall or first run depending on setup).

Yes. Use the TypeScript template from apify create. Types ship with apify and crawlee packages for request handlers, datasets, and inputs.

By default, dataset items are written under storage/datasets/default as JSON files. This mirrors how datasets behave in the cloud, which makes local debugging predictable.

Use the current Node LTS (18 or 20+) to match Apify base images and avoid EOL runtimes. Pin the version in package.json engines if you collaborate in a team.

Define INPUT_SCHEMA.json in the project root for the Console UI. In code, read const input = await Actor.getInput() after Actor.init(). Use defaults when input is undefined for local tests.

No. Apify provides a Python SDK as well. This page covers JavaScript; see the Crawlee Python tutorial linked in Next steps for the other path.

Quick Answer​

Prerequisites​

Step 1: Install the Apify CLI​

Step 2: Create a project​

Step 3: Minimal Actor skeleton​

Step 4: Add Crawlee + Playwright crawling​

Step 5: Run locally​

Step 6: Deploy with apify push​

Core concepts map​

Next steps​