YouTube Knowledge Extractor

Role: Solo, end-to-end
Tech Stack: Python, yt-dlp, OpenAI Whisper API, OpenAI API, Pydantic, ThreadPoolExecutor
Repository: Private. Demo available on request

Overview

YouTube Knowledge Extractor turns YouTube content into structured, queryable knowledge graphs. Instead of generating passive summaries, it extracts domain-specific information (QA pairs, learning objectives, ingredient lists) and outputs fine-tuning datasets.

The system handles both single videos and entire channels, producing training corpora that actually capture a creator’s domain expertise rather than just paraphrasing it.

Phase 1: The Foundation Architecture

The current iteration is a “video-to-markdown” pipeline that handles the real engineering problems: audio processing at scale and API constraints.

The Ingestion Layer

The core pulls audio with yt-dlp directly to MP3. A common pitfall in audio processing is the 25MB file size limit imposed by OpenAI’s Whisper API. To handle long-form content (documentaries, lectures), I implemented ffmpeg splitting logic that cuts audio into 5-minute chunks without re-encoding, which preserves audio quality and processing speed.

Parallelism & Performance

Processing a 3-hour video sequentially is painfully slow. I used Python’s ThreadPoolExecutor to transcribe chunks in parallel.

# A simplified look at the parallel execution strategy
with ThreadPoolExecutor() as executor:
    future_to_chunk = {
        executor.submit(transcribe_chunk, chunk_filename): i
        for i, chunk_filename in enumerate(chunk_filenames)
    }
    # Results are re-assembled in order based on index

Transcription speed ends up constrained by API rate limits rather than local compute, which is exactly the constraint you want.

The “Prompt Strategy” Pattern

To make the analysis useful, I moved away from generic “summarize this” prompts. I built a library of prompt templates stored in prompts.py. Depending on the user’s intent, the system swaps the system prompt context:

Modularity like this means the tool can produce domain-specific structured output instead of generic summaries.

Solved: The Translation Context Problem

One of the trickier engineering problems was translation. Translating chunk-by-chunk kills context, a sentence starts in chunk A and ends in chunk B, and you lose the thread.

So I implemented a context-preserving workflow. It first transcribes the full audio in the source language (e.g., Farsi), reassembles it, then sends larger context windows to an LLM for translation. Idioms and sentence structures survive the translation process.

Phase 2: The Extraction Engine

The system focuses on bulk processing for building AI training datasets.

1. Structured Output over Markdown

Instead of human-readable Markdown, the engine uses Pydantic and the Instructor library to force LLMs to output valid JSON. It extracts:

2. Test-Driven Data Quality

The testing strategy includes:

System Architecture

graph TD
    A[YouTube Video] -->|yt-dlp| B[Audio Extract]
    B -->|Whisper API| C[Raw Transcript]
    C -->|Chunking| D[Transcript Segments]
    D -->|Parallel Processing| E[Content Analysis]
    E -->|Pydantic Schemas| F[Structured Extraction]
    F -->|Knowledge Graph| G[Fine-Tuning Dataset]
    G --> H[Domain Expert Model]

Engineering Details

1. Parallel Transcription with Rate Limiting

Long-form content (3+ hour lectures) is painfully slow to process sequentially because of API rate limits. Parallel chunk processing with ThreadPoolExecutor solves it.

2. Context-Preserving Translation

Translating chunk-by-chunk kills context when sentences split across chunks. Two-pass translation strategy handles it.

3. Domain-Specific Prompt Templates

Generic summarization doesn’t capture domain-specific value. The Prompt Strategy Pattern with specialized templates handles different content types:

Output & Use Cases

Tech Stack