# aparimeyatiwari/msq_bot_podcast architecture

> Architecture diagram of the GitHub repository aparimeyatiwari/msq_bot_podcast, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-15). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aparimeyatiwari/msq_bot_podcast
- Repository: https://github.com/aparimeyatiwari/msq_bot_podcast

## Overview

##### Project Type and Purpose
- This is a full-stack AI-driven Podcast Question Generator.
- The frontend is a simple Flask-based web UI (upload a PDF or transcript, view generated questions).
- The core backend logic lives in the AI_Generator (and mirrored in my_package) Python modules that orchestrate several Ollama-deployed LLM models (chunking, theming, question generation, NER, formatting).

##### File Structure Analysis
Top-level:
- README.md & requirements.txt – project overview and Python dependencies
- flask_code/ – Flask app (app.py), HTML templates, static assets (CSS), upload & JSON folders
- AI_Generator/ – business-logic package: model wrappers and NLP pipelines
- extraction/ – example inputs & outputs for resume-parsing tests (can be included as sample data)
- my_package/ – duplicate of AI_Generator for packaging or testing
- frontend.py – (optional entry point)

Key patterns:
- Separation between web layer (flask_code/) vs. AI processing (AI_Generator/)
- Configuration via Ollama CLI – each model is loaded into a local Ollama daemon
- File I/O: uploads stored in flask_code/uploads; generated JSON in flask_code/json

##### Diagramming the Architecture
- Client Browser (uploads, views UI)
- Flask Web Server (app.py)
- File Storage (uploads folder, JSON folder)
- AI Service Layer (AI_Generator / my_package modules)
- Ollama Model Server (local Ollama runtime hosting llama3‐based models)
- External APIs (e.g., YouTube Search Agent if used)

b. Relationships and Data Flow:
1. User → Browser → HTTP POST (PDF upload) → Flask endpoint
2. Flask writes file to “uploads/”, invokes AI_Generator pipeline
3. AI_Generator:
- resume_parse.py or yt_search_agent.py reads file / fetches transcript
- text_ner.py performs named‐entity recognition via model “custom_ner_model”
- chunk_theme_model & final_theme_model produce thematic segments
- question_generation_model generates question set
- format_model post-processes into JSON/text
4. For each model call, AI_Generator makes an Ollama API/CLI call → Ollama runtime
5. Processed JSON written back to flask_code/json/
6. Flask endpoint returns JSON to client → Browser renders page2.html

c. Architectural Patterns & Principles:
- Layered architecture: Presentation (Flask templates), Application (Flask routes), Domain/AI (AI_Generator modules), Infrastructure (Ollama, file system)
- Single Responsibility: Each model wrapper module handles one task (NER, chunking, theming, Q-gen)
- Externalize LLMs: Models run in Ollama, decoupling heavy AI inference from Python code

d. Key Technologies / Frameworks:
- Python 3 + Flask
- Ollama CLI & llama3 models
- Jinja2 templates, static CSS
- Local file storage (uploads, JSON)
- (Optionally) YouTube Data API via yt_search_agent.py

##### Critical Details to Capture
- Each LLM model’s role and Ollama mapping (e.g., “question_generation_model → podcast_model”)
- The sequential pipeline: parse → NER → chunk/theme → question gen → format
- Where files live on disk vs. in-memory transfer
- Entry points: app.py endpoints, questions_gen.py functions

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Client Layer
    subgraph "Client" 
        style Client fill:#CEF6FF,stroke:#000
        Browser["Client Browser"]:::frontend
    end

    %% Web Server Layer
    subgraph "Web Server (Flask)" 
        style WebServer fill:#CEF6FF,stroke:#000
        App["flask_code/app.py"]:::backend
        Template1["index.html"]:::frontend
        Template2["page2.html"]:::frontend
        CSS1["style.css"]:::frontend
        CSS2["style2.css"]:::frontend
    end

    %% File Storage Layer
    subgraph "File Storage" 
        style FileStorage fill:#D8D8D8,stroke:#000
        Uploads["uploads/"]:::storage
        JSONOut["json/"]:::storage
    end

    %% AI Processing Layer
    subgraph "AI Service Layer" 
        style AIService fill:#D0F5A9,stroke:#000
        AIPkg["AI_Generator/"]:::backend
        ResumeParse["resume_parse.py"]:::backend
        YTAgent["yt_search_agent.py"]:::backend
        NER["text_ner.py"]:::backend
        ThemeGen["theme_generation.py"]:::backend
        ChunkModel["chunk_theme_model/"]:::backend
        FinalTheme["final_theme_model/"]:::backend
        QGenPipeline["questions_gen.py"]:::backend
        QModel["question_generation_model/"]:::backend
        FormatModel["format_model/"]:::backend
        NerModel["ner_model/"]:::backend
    end

    %% Ollama and External APIs
    Ollama["Ollama Model Server"]:::ai
    ExternalAPI["YouTube Data API"]:::external
    SampleData["extraction/input_resume/"]:::storage

    %% Optional Entry
    FrontendScript["frontend.py"]:::backend

    %% Connections
    Browser -->|HTTP POST upload| App
    App -->|writes file| Uploads
    App -->|invokes pipeline| AIPkg
    AIPkg --> ResumeParse
    AIPkg --> YTAgent
    AIPkg --> NER
    AIPkg --> ThemeGen
    AIPkg --> ChunkModel
    AIPkg --> FinalTheme
    AIPkg --> QGenPipeline
    AIPkg --> QModel
    AIPkg --> FormatModel
    AIPkg --> NerModel

    ResumeParse -->|reads file| Uploads
    YTAgent -->|fetch transcript| ExternalAPI
    NER -->|calls model| Ollama
    ThemeGen -->|calls model| Ollama
    ChunkModel -->|calls model| Ollama
    FinalTheme -->|calls model| Ollama
    QGenPipeline -->|calls model| Ollama
    QModel -->|calls model| Ollama
    FormatModel -->|post-process| JSONOut

    App -->|returns JSON| JSONOut
    JSONOut -->|HTTP response| Browser

    ExternalAPI -->|optional data| AIPkg
    SampleData -->|example input| AIPkg
    FrontendScript -.-> App

    %% Click Events
    click App "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/flask_code/app.py"
    click Template1 "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/flask_code/templates/index.html"
    click Template2 "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/flask_code/templates/page2.html"
    click CSS1 "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/flask_code/static/style.css"
    click CSS2 "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/flask_code/static/style2.css"
    click Uploads "https://github.com/aparimeyatiwari/msq_bot_podcast/tree/main/flask_code/uploads/"
    click JSONOut "https://github.com/aparimeyatiwari/msq_bot_podcast/tree/main/flask_code/json/"
    click AIPkg "https://github.com/aparimeyatiwari/msq_bot_podcast/tree/main/AI_Generator/"
    click ResumeParse "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/AI_Generator/resume_parse.py"
    click YTAgent "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/AI_Generator/yt_search_agent.py"
    click NER "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/AI_Generator/text_ner.py"
    click ThemeGen "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/AI_Generator/theme_generation.py"
    click ChunkModel "https://github.com/aparimeyatiwari/msq_bot_podcast/tree/main/AI_Generator/chunk_theme_model/"
    click FinalTheme "https://github.com/aparimeyatiwari/msq_bot_podcast/tree/main/AI_Generator/final_theme_model/"
    click QGenPipeline "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/AI_Generator/questions_gen.py"
    click QModel "https://github.com/aparimeyatiwari/msq_bot_podcast/tree/main/AI_Generator/question_generation_model/"
    click FormatModel "https://github.com/aparimeyatiwari/msq_bot_podcast/tree/main/AI_Generator/format_model/"
    click NerModel "https://github.com/aparimeyatiwari/msq_bot_podcast/tree/main/AI_Generator/ner_model/"
    click SampleData "https://github.com/aparimeyatiwari/msq_bot_podcast/tree/main/extraction/input_resume/"
    click FrontendScript "https://github.com/aparimeyatiwari/msq_bot_podcast/blob/main/frontend.py"

    %% Styles
    classDef frontend fill:#AED6F1,stroke:#21618C,color:#000
    classDef backend fill:#ABEBC6,stroke:#196F3D,color:#000
    classDef ai fill:#F9E79F,stroke:#B9770E,color:#000
    classDef storage fill:#D5DBDB,stroke:#424949,color:#000
    classDef external fill:#E8DAEF,stroke:#7D3C98,color:#000
```
