Skip to main content

Web Scraping for B2B Sales Enrichment: Build a Lead Research Pipeline (2026)

· 7 min read
Yassine El Haddad
Software Developer & Automation Specialist

I build production AI agents, web scrapers, and automation pipelines. Most of what I publish here comes from the actual problems they run into: proxies that get banned, anti-bot stacks that fingerprint your client, RAG that drifts when the underlying data moves. Stack: Python, TypeScript, Go, FastAPI, LangChain, Crawlee, Playwright, deployed on AWS, GCP, and Cloudflare.

B2B sales enrichment means turning a bare company name or domain into a complete prospect record: employee count, tech stack, revenue signals, recent job postings, decision-maker names, and contact information.

Traditional data providers (ZoomInfo, Apollo) cost $10,000–$50,000/year. Web scraping builds the same enrichment pipeline for a fraction of the cost — and stays current automatically.

What Data Can You Enrich Via Scraping?​

Data PointSourceTool
Company description, industryCompany websiteApify/Firecrawl
Employee countLinkedInApify LinkedIn Actor
Tech stackBuiltWith, WappalyzerAPI or scraper
Funding/valuationCrunchbaseApify Crunchbase Actor
Recent job postingsLinkedIn, IndeedApify Job Scraper
Decision-maker names/titlesLinkedInApify LinkedIn People Search
Company email formatHunter.ioHunter API
Social proof (reviews)G2, CapterraApify review scrapers
Google Maps ratingGoogle MapsApify Google Maps Actor

Pipeline Architecture​

[Input: domain/company list]
↓
[Step 1: Website scrape → description, location, phone]
↓
[Step 2: LinkedIn company page → employee count, industry, specialties]
↓
[Step 3: Job postings → growth signals, tech requirements]
↓
[Step 4: Crunchbase → funding round, investor data]
↓
[Step 5: AI enrichment → ICP score, personalized pitch angle]
↓
[Output: Enriched CRM record]

Step 1: Scrape Company Websites​

import requests

def enrich_from_website(domain: str) -> dict:
"""Scrape company website for key data."""
resp = requests.post(
"https://api.firecrawl.dev/v1/scrape",
json={
"url": f"https://{domain}",
"formats": ["extract"],
"extract": {
"prompt": "Extract: company description (1-2 sentences), primary product/service, target customer, company location, contact email if visible.",
"schema": {
"type": "object",
"properties": {
"description": {"type": "string"},
"product": {"type": "string"},
"target_customer": {"type": "string"},
"location": {"type": "string"},
"email": {"type": "string"}
}
}
}
},
headers={"Authorization": f"Bearer {FIRECRAWL_KEY}"}
)
return resp.json().get("data", {}).get("extract", {})

Step 2: LinkedIn Company Enrichment via Apify​

LinkedIn company pages give you the firmographics a website rarely states: headcount, industry, specialities, founding year and headquarters. The LinkedIn Company Details Scraper takes company page URLs (or plain company names) and needs no LinkedIn cookies or login. It costs $0.004/company subject to change · verified 2026-10-23 on the Free plan, plus a $0.00005 start fee per run.

// LinkedIn company pages -> firmographics (harvestapi/linkedin-company, no cookies)
const response = await fetch(
'https://api.apify.com/v2/acts/harvestapi~linkedin-company/run-sync-get-dataset-items',
{
method: 'POST',
headers: {
'Authorization': `Bearer ${APIFY_TOKEN}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
// Company page URLs. If you only have names, pass them as `searches` instead.
companies: companies.map(c => `https://www.linkedin.com/company/${c.linkedinHandle}/`)
})
}
);

const items = await response.json();
const firmographics = items.map(item => ({
linkedinUrl: item.linkedinUrl,
website: item.website,
employeeCount: item.employeeCount,
employeeRange: item.employeeCountRange, // { start, end }
industry: item.industries?.[0]?.name,
specialities: item.specialities,
foundedYear: item.foundedOn?.year,
headquarters: item.locations?.find(l => l.headquarter)
}));

run-sync-get-dataset-items returns the results in the same request, but it waits at most 300 seconds. For long company lists, start the run with /runs and read the dataset when it finishes.

Real output from a test run on 2026-10-23 (two companies, trimmed). The run cost $0.00805:

FieldApifyVercel
employeeCount2741,029
employeeCountRange201–500501–1,000
industries[0].nameTechnology, Information and InternetSoftware Development
specialities (first 3)Web scraping, Browser automation, AI agentsNext.js, Developer Velocity, Open Source
foundedOn.year20162015
HQ city (locations, headquarter: true)PragueSan Francisco

Note that the two size fields can disagree: Vercel's employeeCount sits above its own employeeCountRange band. Pick one field for ICP scoring and use it consistently.

Keep this step to company pages. LinkedIn's User Agreement prohibits automated scraping, and company firmographics are business data, not personal data, which keeps you clear of most GDPR questions. Read the legal background on LinkedIn scraping before you scale it.


Step 3: Job Posting Signals​

Job postings are one of the strongest buying signals in B2B sales:

  • Hiring a Head of Security → ICP for security tools
  • Hiring Data Engineers → ICP for data infrastructure
  • Hiring Account Executives → growing sales team = budget to spend
// Actor: https://apify.com/curious_coder/linkedin-jobs-scraper (input fields checked 2026-10-21)
const jobResponse = await fetch(
'https://api.apify.com/v2/acts/curious_coder~linkedin-jobs-scraper/runs',
{
method: 'POST',
headers: {
'Authorization': `Bearer ${APIFY_TOKEN}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
keywords: 'data engineer',
location: 'San Francisco',
datePosted: 'pastMonth',
limitPerSource: 20,
})
}
);

The Actor has no company-name filter. To watch a specific account, pass its LinkedIn company ID in companyIds, or filter the results by company name afterwards. LinkedIn Jobs Scraper bills per job returned.

Analyze job titles and descriptions to infer tech stack and growth stage. To watch a named account list for hiring, funding and tech changes every month, see buying signals with Claude.


Step 4: AI Scoring with Claude​

Use Claude to score each enriched record against your ICP and generate a personalized pitch:

import anthropic

client = anthropic.Anthropic()

def score_and_pitch(enriched_record: dict) -> dict:
response = client.messages.create(
model="claude-3-7-sonnet-20250219",
max_tokens=1000,
messages=[{
"role": "user",
"content": f"""
Company data: {enriched_record}

Our ICP: SaaS companies with 50-500 employees, growing data team,
headquartered in North America or Europe.

1. Score this company against our ICP (0-10)
2. Write a 2-sentence personalized email opening referencing specific data above
3. Identify the most relevant pain point we can address

Return JSON with fields: icp_score, email_opener, pain_point
"""
}],
)
return response.content[0].text

Step 5: Automate with Make.com​

Use Make.com to orchestrate the full pipeline on autopilot:

[New row in Google Sheets]
→ [HTTP: Run Apify Website Scraper]
→ [HTTP: Run Apify LinkedIn Actor]
→ [Claude: Score + pitch]
→ [HTTP: Push to HubSpot/Salesforce]
→ [Slack: Alert sales rep]

Total cost per lead: ~$0.10–$0.30 using Apify + Claude + Make.com.


Cost Comparison​

MethodSetup CostPer LeadFreshness
ZoomInfo~$15,000/year$0 (covered)Monthly
Apollo$99–$249/month~$0.10–0.50Weekly
This pipeline~$50–$100/month$0.10–0.30Real-time

The scraping pipeline wins on freshness (scraped on demand) and total cost at scale.

Start with Apify's free tier → | Make.com free trial →


FAQ​

Frequently Asked Questions

The three highest-quality sources are: (1) Google Maps for local business discovery with address, phone, website, and ratings; (2) LinkedIn for decision-maker identification (name, title, company, contact signals); and (3) company websites for direct email addresses, technographic signals (job board postings, tech stack), and current news. Combining all three gives you the most complete B2B lead profile.

Scraping publicly visible business information (company name, phone, website, address from Google Maps and company websites) is widely practiced for B2B lead generation. LinkedIn's ToS restricts automated access; any LinkedIn data collection must respect rate limits and avoid storing personal data beyond its intended purpose. GDPR and CCPA require you to have a legitimate purpose for storing individual contact data. Always validate your use case with legal counsel for regulated industries.

LinkedIn does not expose email addresses through scraping — you can extract name, title, company, and profile URL. To get email addresses, enrich the LinkedIn profile against Apollo.io, Hunter.io, or similar email finders using the company domain as an input. Apify's Contact Details Scraper can also extract emails from company websites directly.

The total cost is approximately $0.10–$0.30 per lead using Apify (Google Maps + website scraping) + Claude API (scoring/pitch generation) + Make.com (automation). At 500 leads/month, that is $50–$150 total — significantly cheaper than ZoomInfo ($15K/year) or Apollo ($99–$249/month) with better data freshness since you scrape on demand.

Common mistakes and fixes

LinkedIn scraping returns limited data or gets blocked.

LinkedIn aggressively blocks automation. Use Apify's LinkedIn actors which handle session rotation. Alternatively, use Bright Data's LinkedIn Dataset for compliance-safe bulk extraction.

Company website scraping misses key data for many companies.

Add fallback data sources: scrape the Google My Business listing, the company's LinkedIn page, and their Crunchbase profile for funding/employee data.

Enrichment pipeline is too slow for large lead lists.

Run Apify Actors in parallel using the API. Batch 50 companies per Actor run, and trigger multiple parallel runs using Make.com's parallel routing.

Advertise on this page