YouTube Transcripts for LLM and RAG Pipelines (2026)
The underused RAG corpus
Most RAG pipelines ingest PDFs, web pages, and documentation. Few teams tap into YouTube — and that is a significant gap. YouTube hosts decades of expert spoken content across every domain: medical lectures, financial analysis, engineering walkthroughs, legal commentary, academic conference talks. This content does not exist as text anywhere else.
A single channel from a domain expert can represent thousands of hours of structured knowledge. Turned into transcripts, it becomes a high-quality, domain-specific retrieval corpus for an LLM that no public dataset provides.
YouTube Transcript Scraper Pro extracts that corpus at scale — with a transcript_llm field designed for direct ingestion into any RAG framework.
The transcript_llm field
Standard transcript text includes tokens that consume context window budget without adding meaning:
[Music] Welcome back everyone so today we're going to [Applause] um talk about
[Music] the fundamentals of uh machine learning and I think [Applause]
The transcript_llm field strips these:
Welcome back everyone so today we're going to talk about the fundamentals of
machine learning and I think
Tokens stripped include: [Music], [Applause], [Laughter], (laughs), [Inaudible], stage directions, and other non-speech annotations. Whitespace is normalised. The result passes directly to a vector store without pre-processing.
To request it, add "llm" to outputFormats:
run_input = {
"startUrls": [{"url": "https://www.youtube.com/@channelname"}],
"outputFormats": ["llm"],
"maxResults": 50,
}
LangChain integration
from apify_client import ApifyClient
from langchain_core.documents import Document
from langchain_openai import OpenAIEmbeddings
from langchain_chroma import Chroma
client = ApifyClient("YOUR_API_TOKEN")
run = client.actor("codepoetry/youtube-transcript-ai-scraper").call(
run_input={
"startUrls": [{"url": "https://www.youtube.com/playlist?list=YOUR_PLAYLIST_ID"}],
"outputFormats": ["llm"],
"languages": ["en"],
"enableAiFallback": True,
"maxAiMinutes": 120,
}
)
docs = [
Document(
page_content=item["transcript_llm"],
metadata={
"source": item["metadata"]["url"],
"title": item["metadata"]["title"],
"channel": item["metadata"]["channel"],
"duration_sec": item["metadata"]["duration"],
"upload_date": item["metadata"]["upload_date"],
"is_ai_generated": item.get("is_ai_generated", False),
},
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items()
if item.get("transcript_llm")
]
vectorstore = Chroma.from_documents(docs, OpenAIEmbeddings())
retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
Install dependencies: pip install apify-client langchain-core langchain-openai langchain-chroma
The is_ai_generated metadata flag lets you filter or weight results by source type downstream.
LlamaIndex integration
from apify_client import ApifyClient
from llama_index.core import Document, VectorStoreIndex
client = ApifyClient("YOUR_API_TOKEN")
run = client.actor("codepoetry/youtube-transcript-ai-scraper").call(
run_input={
"startUrls": [{"url": "https://www.youtube.com/@expertchannel"}],
"outputFormats": ["llm"],
"languages": ["en"],
"enableAiFallback": False,
"maxResults": 100,
}
)
documents = [
Document(
text=item["transcript_llm"],
metadata={
"url": item["metadata"]["url"],
"title": item["metadata"]["title"],
"channel": item["metadata"]["channel"],
},
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items()
if item.get("transcript_llm")
]
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("Explain the main argument from the most recent video")
Chunking strategy
YouTube segments can be minutes long. Chunking by segment boundary produces uneven, context-poor chunks. Two better strategies:
Sentence-boundary chunking (recommended for most pipelines)
Split transcript_llm at sentence boundaries using nltk.sent_tokenize or spacy. Aim for 200–400 token chunks with 20–40 token overlap. This produces semantically coherent units independent of caption timing.
Segment-boundary chunking (when timestamps matter)
Use transcript_json instead of transcript_llm. Each segment object includes start, end, and text. Group segments into fixed-duration windows (e.g., 60 seconds) and store timestamps in metadata for source attribution in citations.
# Segment-window chunking example
segments = item["transcript_json"]
window_sec = 60
chunks = []
current, current_start = [], segments[0]["start"]
for seg in segments:
current.append(seg["text"])
if seg["end"] - current_start >= window_sec:
chunks.append({
"text": " ".join(current),
"start": current_start,
"end": seg["end"],
"url": f"{item['metadata']['url']}&t={int(current_start)}",
})
current, current_start = [], seg["end"]
if current:
chunks.append({"text": " ".join(current), "start": current_start, "end": seg["end"]})
The &t= URL parameter lets you link citations back to the exact timestamp in the original video.
Batch ingestion from a channel or playlist
No code? The same channel-to-corpus pipeline runs inside Claude: chat with a YouTube channel in Claude, with every answer cited to a video and timestamp.
For large corpus builds, use a channel URL and maxResults to control scope:
run_input = {
"startUrls": [{"url": "https://www.youtube.com/@yourexpertchannel"}],
"outputFormats": ["json", "llm"],
"languages": ["en"],
"enableAiFallback": True,
"maxAiMinutes": 300,
"skipAiFallbackIfLongerThan": 90,
"maxResults": 200,
}
Before running on an unknown channel, cap the AI transcription budget with the run-level knobs:
cost_control = {
"maxAiMinutes": 300, # AI budget per run (default 30, max 600)
"skipAiFallbackIfLongerThan": 90, # skip AI fallback for videos longer than 90 minutes
}
maxAiMinutes sets how many minutes of AI transcription a single run may spend — the default is 30 and the maximum is 600. skipAiFallbackIfLongerThan skips the AI fallback for long videos so a single stream does not drain the budget, and videos longer than 120 minutes never go through AI fallback at all.
Cost to build a corpus
Pricing on the free Apify plan:
| Corpus size | Videos | AI needed | Estimated cost |
|---|---|---|---|
| Small (channel sample) | 50 | No | ~$0.053 |
| Medium (full channel) | 200 | No | ~$0.203 |
| Large (mixed — 20% no captions, avg 10 min) | 500 | Mixed | ~$12.50 |
| Podcast archive (all AI, avg 45 min) | 100 | Yes — 4,500 min | ~$54.10 |
On the free $5 credit: approximately 5,000 native transcripts or 400 minutes of AI transcription. For mixed corpora, native captions are always checked first — AI cost only applies to videos that need it.
See the actor pricing page for plan discounts.
Word-level timestamps
Enable wordLevel: true to receive per-word timestamps in transcript_json:
{
"start": 18.5,
"end": 21.0,
"text": "We're no strangers to love",
"words": [
{ "start": 18.5, "end": 18.8, "text": "We're" },
{ "start": 18.8, "end": 19.1, "text": "no" },
{ "start": 19.1, "end": 19.6, "text": "strangers" }
]
}
For auto-generated captions and AI transcriptions, the per-word values are derived from segment timing — useful structure for audio alignment or citation linking, but treat them as approximate rather than a guaranteed field. Manual captions carry no word timing.
Try YouTube Transcript Scraper Pro →
Sign up for Apify — free credits on every new account.
The built-in speech-to-text model performs well on clear speech in widely spoken languages. Accuracy degrades with heavy accents, domain jargon, or poor audio quality. The language_probability field (0–1) indicates the model's confidence in language detection. For quality-critical corpora, treat AI transcripts as a first draft and plan a review pass for high-value content.
99 languages including English, Spanish, French, German, Portuguese, Japanese, Chinese, Arabic, and Hindi. Pass forceTranscriptionLanguage (ISO 639-1 code) to skip the 30-second auto-detection window — this reduces processing time by ~20% when you know the channel language in advance.
If all 500 videos have native captions, the cost on the free plan is approximately $0.50 (500 × $0.001 + $0.0025 start fee). If 20% have no captions and average 10 minutes each, add 1,000 AI minutes × $0.012 = $12.00, for a total of ~$12.50. Use skipAiFallbackIfLongerThan to keep long videos from inflating the AI bill.
Yes. Set wordLevel: true to add a words array to each transcript_json segment. Each word has start, end, and text fields — treat the per-word values as approximate, since they are derived from segment timing. Available for auto-generated captions and AI transcriptions — not manual captions.
No. The built-in speech-to-text model runs on Apify's compute as part of the actor. You pay through Apify's Pay-Per-Event pricing ($0.012/min on the free plan) — no separate speech-to-text API account or OpenAI key is needed.
