Skip to main content

Apify Actor Testing Guide 2026: Unit, Integration & E2E Tests

· 4 min read
Yassine El Haddad
Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

Test your Apify Actors before they run in production. Use unit tests for extraction logic, integration tests for live URLs, and apify run for end-to-end validation. Test in the Apify Console or run locally.

Why test Actors?

Actors run on remote infrastructure. A small logic bug can waste compute credits or produce bad data. Unit tests catch extraction bugs. Integration tests catch blocking and rate limits. Local runs with apify run verify the full pipeline before you push.

Unit testing extraction logic

Extract the parsing logic into pure functions. Test them with Jest or Vitest—no network, no Apify SDK.

// extractors/product.ts
export function extractProductPrice($: CheerioAPI): number | null {
const raw = $('.price').first().text().replace(/[^0-9.]/g, '');
return raw ? parseFloat(raw) : null;
}
// extractors/product.test.ts
import * as cheerio from 'cheerio';
import { extractProductPrice } from './product';

describe('extractProductPrice', () => {
it('parses USD price', () => {
const html = '<span class="price">$19.99</span>';
const $ = cheerio.load(html);
expect(extractProductPrice($)).toBe(19.99);
});

it('returns null when no price found', () => {
const $ = cheerio.load('<div>No price</div>');
expect(extractProductPrice($)).toBeNull();
});
});

Keep extraction pure. Pass in HTML or a Cheerio instance. No Actor.pushData in unit tests.

Mocking the Apify SDK

For logic that calls Actor.pushData or Actor.getInput, mock the SDK.

import { Actor } from 'apify';

jest.mock('apify', () => ({
Actor: {
init: jest.fn(),
getInput: jest.fn(),
pushData: jest.fn(),
exit: jest.fn(),
},
}));

beforeEach(() => {
(Actor.getInput as jest.Mock).mockResolvedValue({ startUrls: [{ url: 'https://example.com' }] });
(Actor.pushData as jest.Mock).mockResolvedValue(undefined);
});

Integration tests with Crawlee

Use Crawlee's test utilities or a real crawler with a small, stable test URL. Keep runs short.

// integration/crawler.test.ts
import { CheerioCrawler } from 'crawlee';

it('scrapes a known test page', async () => {
const results: Record<string, unknown>[] = [];
const crawler = new CheerioCrawler({
maxRequestsPerCrawl: 2,
async requestHandler({ $, pushData }) {
const title = $('title').text();
await pushData({ title });
},
});
await crawler.run(['https://example.com']);
// Assert via crawler internals or a shared store
}, 15000);

Use maxRequestsPerCrawl to cap requests. Prefer static HTML fixtures when possible.

Local runs with apify run

The Apify CLI runs your Actor locally as if it were on the platform. Input comes from .actor/actor.json defaultInput or --input:

apify run
apify run --input '{"startUrls":[{"url":"https://example.com"}]}'

Verify dataset output in storage/datasets/default/. Use this as your final smoke test before pushing.

Testing with mock datasets

If your Actor reads from a dataset (e.g. for post-processing), seed storage/datasets/default/ with test JSON:

mkdir -p storage/datasets/default
echo '[{"url":"https://example.com","title":"Test"}]' > storage/datasets/default/000000001.json
apify run

CI/CD integration

Run tests in CI before every push. Example GitHub Actions:

- run: npm ci
- run: npm test # unit + integration
- run: apify run # optional: full local run (slower)

Gate deployments on passing tests. Use apify push only after apify run succeeds.

Common Actor bugs to test for

Bug typeTest approach
Selector breakageUnit test extractors with real HTML snippets from target
Empty/malformed inputTest with required fields missing, invalid URLs
Proxy/session misuseIntegration test with a small set of URLs
Rate limit handlingMock 429 responses, assert retry/backoff behavior
Output schema driftSnapshot test or schema validation on pushData payloads

Comparison: testing strategies

StrategySpeedNetworkBest for
Unit (Jest/Vitest)FastNoneExtraction, parsing
Integration (Crawlee)MediumYesAnti-bot, selectors
apify runSlowerYesFull pipeline smoke
CI apify runSlowestYesPre-deploy check

Start with unit tests. Add integration tests for critical paths. Use apify run before each release.

Apify Affiliate Banner 728x90Apify Affiliate Banner 728x90Apify Affiliate Banner 300x50Apify Affiliate Banner 300x50
Next step

Add unit tests for one extractor, then run apify run locally. Deploy with confidence →

Frequently Asked Questions

Use apify run from the Actor directory. It executes your code locally with Apify storage emulation. Pass input via --input or defaultInput in .actor/actor.json.

Yes. Extract parsing logic into pure functions that accept HTML or a Cheerio instance. Test them with Jest or Vitest. Mock Actor.pushData and Actor.getInput when testing orchestration code.

Use jest.mock for fetch/got, or pass pre-captured HTML to your extractors. For Crawlee, use RequestQueue with fake responses or point at a static test site.

Yes. Use apify push from CI after tests pass. Run npm test for unit/integration tests and apify run for a full local smoke test before pushing.

Common mistakes and fixes

apify run fails with missing env vars

Set APIFY_TOKEN and APIFY_ACTOR_ID in .env or run apify login first.

Mocked Actor.pushData doesn't match runtime behavior

Use Actor.on('pushData', handler) or verify pushData was called with correct shape.