# gabrielchua/open-notebooklm architecture

> Architecture diagram of the GitHub repository gabrielchua/open-notebooklm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gabrielchua/open-notebooklm
- Repository: https://github.com/gabrielchua/open-notebooklm

## Overview

##### Project Type and Purpose
- This project is an open-source tool that transforms PDF content into engaging podcast dialogues. It functions as a full-stack application with a lightweight Gradio-based interface for users.
- The core idea is to process a PDF file, use open-source LLMs (e.g., Llama 3.3 70B via Fireworks API) to generate natural dialogue, and then convert this dialogue into speech (using TTS models like MeloTTS and Bark).

##### Analyzing the File Structure
- Key Files:
- app.py: Likely the entry point for the Gradio web interface.
- constants.py, prompts.py, schema.py, and utils.py: These files suggest separation of concerns in handling configuration/constants, prompt generation for the LLM, data validation or schema definitions, and utility functions for tasks such as PDF processing.
- examples/ and examples_cached/: These directories provide sample PDFs and possibly processed versions, useful for testing or demonstration.
- GitHub configurations (in .github/workflows) ensure that the project’s deployment and testing pipelines are set up properly (e.g., syncing with Hugging Face).

a. Main Components:
- User Interface (Gradio Interface): The front-facing component where users upload their PDFs.
- PDF Processor: A module (likely within utils.py) responsible for reading and extracting text from PDFs.
- Dialogue Generator: Modules that interact with the LLM (using prompts.py and constants.py) through the Fireworks API. This incorporates the Llama model along with any instructor components.
- Text-to-Speech Module: Integrates TTS engines such as MeloTTS and Bark to convert generated dialogue into MP3 audio files.
- External Integrations/Dependencies:
- Fireworks API hosting LLaMA 3.3 70B
- External TTS services/models
- Optionally, the Jina Reader for content handling

b. Relationships and Interactions:
- Data Flow:
1. A user uploads a PDF via the Gradio interface.
2. The PDF Processor reads and extracts content from the file.
3. The extracted content is formatted into a prompt (possibly with the help of prompts.py) and sent to the LLM via the Fireworks API.
4. The generated dialogue is then passed to the Text-to-Speech Module.
5. Finally, the TTS module converts the text into an MP3 file ready for download or streaming.
- Dependency Flow:
- Libraries and API keys (like FIREWORKS_API_KEY) are embedded in the environment and accessed by the application logic.

c. Highlighting Architectural Patterns / Design Principles:
- Separation of concerns is clearly observed by splitting PDF processing, dialogue generation, and TTS conversion into separate modules/files.
- The design may follow a pipeline or layered architecture, ensuring that input processing, business logic, and output generation are distinct.
- Use of external services (API for LLM and TTS engines) indicates reliance on a service-oriented architecture for those components.

d. Relevant Technologies and Frameworks:
- Gradio (for the interface)
- Python (as the primary language)
- Fireworks API & Llama (for language modeling)
- MeloTTS and Bark (as TTS providers)
- Possibly Jina Reader for document reading
- GitHub Actions (for CI/CD as seen in the .github/workflows folder)

## Diagram (Mermaid)

```mermaid
graph TD
    %% User Interaction
    subgraph "User Interaction"
        UI["User Interface: Gradio"]:::interface
        Audio["Audio Output (MP3)"]:::output
    end

    %% Application Logic
    subgraph "Application Logic"
        PDF["PDF Processor"]:::processor
        subgraph "Dialogue Generator"
            DG["Core Dialogue Module"]:::generator
            P["Prompts"]:::generatorDetail
            C["Constants"]:::generatorDetail
            S["Schema"]:::generatorDetail
        end
        TTS["TTS Processor"]:::tts
    end

    %% External Services
    subgraph "External Services"
        LLM["Fireworks API (Llama 3.3 70B)"]:::external
        TTSService["TTS Engines (MeloTTS/Bark)"]:::external
    end

    %% Supporting Resources
    subgraph "Supporting Resources"
        E["Examples"]:::storage
        EC["Examples Cached"]:::storage
    end

    %% DevOps
    subgraph "DevOps"
        CI["CI/CD Workflow"]:::devOps
    end

    %% Connections
    UI -->|"upload PDF"| PDF
    PDF -->|"extract text"| DG
    DG -->|"utilizes"| P
    DG -->|"utilizes"| C
    DG -->|"utilizes"| S
    DG -->|"calls LLM"| LLM
    LLM -->|"returns dialogue"| DG
    DG -->|"passes dialogue"| TTS
    TTS -->|"invokes TTS"| TTSService
    TTSService -->|"converts to audio"| TTS
    TTS -->|"generates"| Audio
    PDF ---|"uses sample PDFs"| E
    PDF ---|"uses sample PDFs"| EC

    %% Click Events
    click UI "https://github.com/gabrielchua/open-notebooklm/blob/main/app.py"
    click PDF "https://github.com/gabrielchua/open-notebooklm/blob/main/utils.py"
    click P "https://github.com/gabrielchua/open-notebooklm/blob/main/prompts.py"
    click C "https://github.com/gabrielchua/open-notebooklm/blob/main/constants.py"
    click S "https://github.com/gabrielchua/open-notebooklm/blob/main/schema.py"
    click E "https://github.com/gabrielchua/open-notebooklm/tree/main/examples/"
    click EC "https://github.com/gabrielchua/open-notebooklm/tree/main/examples_cached/"
    click CI "https://github.com/gabrielchua/open-notebooklm/tree/main/.github/workflows/"

    %% Styles
    classDef interface fill:#f9f,stroke:#333,stroke-width:2px;
    classDef processor fill:#bbf,stroke:#333,stroke-width:2px;
    classDef generator fill:#afa,stroke:#333,stroke-width:2px;
    classDef generatorDetail fill:#cfc,stroke:#333,stroke-width:2px;
    classDef tts fill:#fcc,stroke:#333,stroke-width:2px;
    classDef external fill:#fc9,stroke:#333,stroke-width:2px;
    classDef output fill:#ccf,stroke:#333,stroke-width:2px;
    classDef storage fill:#fcf,stroke:#333,stroke-width:2px;
    classDef devOps fill:#c9f,stroke:#333,stroke-width:2px;
```
