# anisayari/topic-modeling-youtube architecture

> Architecture diagram of the GitHub repository anisayari/topic-modeling-youtube, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anisayari/topic-modeling-youtube
- Repository: https://github.com/anisayari/topic-modeling-youtube

## Overview

This is a Flask web app for collecting YouTube channel comments and exploring the saved data. The main workflow is: submit one or more channels, queue extraction, fetch channel/video metadata and comments in parallel, and save progress as per-channel JSON files; users can then inspect channel statistics and video details. The README also describes topic modeling as a planned capability, not an implemented runtime flow. The source excerpt is partial, so some endpoint and worker wiring is based on documented behavior rather than inspected implementation.

Main entry points: [`app.py`](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py) (Flask app and API), [`templates/index.html`](https://github.com/anisayari/topic-modeling-youtube/blob/main/templates/index.html) (Web interface), [`data`](https://github.com/anisayari/topic-modeling-youtube/tree/main/data) (Channel records)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_web["Web interface"]
  node_ui["Web interface<br/>[index.html]"]
  node_server["Flask app and API<br/>[app.py]"]
end

subgraph group_extract["Comment extraction"]
  node_preview["Channel lookup<br/>[app.py]"]
  node_queue["Extraction queue<br/>[app.py]"]
  node_worker["Queue worker<br/>[app.py]"]
  node_channel["Channel metadata<br/>[app.py]"]
  node_videos["Video listing<br/>[app.py]"]
  node_comments["Comment extraction<br/>[app.py]"]
  node_progress["Progress and cancellation<br/>[app.py]"]
end

subgraph group_storage["Data storage"]
  node_channelmeta["Channel records"]
  node_videorecords["Video comment records"]
end

subgraph group_insights["Data insights"]
  node_stats["Channel statistics<br/>[app.py]"]
  node_detail["Channel and video detail<br/>[app.py]"]
  node_charts["Insight visualizations<br/>[index.html]"]
end

subgraph group_topics["Planned topics"]
  node_loading["Data loading"]
  node_preprocessing["Text preprocessing"]
  node_vectorization["Vectorization"]
  node_modeling["Topic algorithms"]
  node_reduction["Dimensionality reduction"]
end

node_user(("User"))
node_youtube{{"YouTube"}}

node_user -->|"uses"| node_ui
node_ui -->|"sends requests"| node_server
node_server -->|"looks up channel"| node_preview
node_preview -->|"queries metadata"| node_youtube
node_server -->|"enqueues channels"| node_queue
node_queue -->|"dispatches jobs"| node_worker
node_worker -->|"resolves channel"| node_channel
node_channel -->|"fetches listing"| node_youtube
node_worker -->|"processes videos"| node_videos
node_videos -->|"fetches video data"| node_youtube
node_worker -->|"extracts comments"| node_comments
node_comments -->|"requests comments"| node_youtube
node_worker -->|"writes metadata"| node_channelmeta
node_worker -->|"saves per video"| node_videorecords
node_server -->|"reports or stops"| node_progress
node_server -->|"serves statistics"| node_stats
node_stats -->|"reads metadata"| node_channelmeta
node_stats -->|"counts comments"| node_videorecords
node_server -->|"serves details"| node_detail
node_detail -->|"reads metadata"| node_channelmeta
node_detail -->|"reads videos"| node_videorecords
node_ui -->|"renders insights"| node_charts
node_loading -.->|"planned stage"| node_preprocessing
node_preprocessing -.->|"planned stage"| node_vectorization
node_vectorization -.->|"planned stage"| node_modeling
node_modeling -.->|"planned stage"| node_reduction

click node_ui "https://github.com/anisayari/topic-modeling-youtube/blob/main/templates/index.html"
click node_server "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_preview "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_queue "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_worker "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_channel "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_videos "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_comments "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_channelmeta "https://github.com/anisayari/topic-modeling-youtube/tree/main/data"
click node_videorecords "https://github.com/anisayari/topic-modeling-youtube/tree/main/data"
click node_progress "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_stats "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_detail "https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py"
click node_charts "https://github.com/anisayari/topic-modeling-youtube/blob/main/templates/index.html"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui,node_server,node_user toneBlue
class node_preview,node_queue,node_worker,node_channel,node_videos,node_comments,node_progress toneAmber
class node_channelmeta,node_videorecords toneMint
class node_stats,node_detail,node_charts toneRose
class node_loading,node_preprocessing,node_vectorization,node_modeling,node_reduction,node_youtube toneIndigo
```

## Components

### Web interface

- [Web interface](https://github.com/anisayari/topic-modeling-youtube/blob/main/templates/index.html): `templates/index.html`
- [Flask app and API](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`

### Comment extraction

- [Channel lookup](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`
- [Extraction queue](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`
- [Queue worker](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`
- [Channel metadata](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`
- [Video listing](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`
- [Comment extraction](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`
- [Progress and cancellation](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`

### Data storage

- [Channel records](https://github.com/anisayari/topic-modeling-youtube/tree/main/data): `data`
- [Video comment records](https://github.com/anisayari/topic-modeling-youtube/tree/main/data): `data`

### Data insights

- [Channel statistics](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`
- [Channel and video detail](https://github.com/anisayari/topic-modeling-youtube/blob/main/app.py): `app.py`
- [Insight visualizations](https://github.com/anisayari/topic-modeling-youtube/blob/main/templates/index.html): `templates/index.html`

### Planned topics

- **Data loading**
- **Text preprocessing**
- **Vectorization**
- **Topic algorithms**
- **Dimensionality reduction**

### Other components

- **User**
- **YouTube**

## Connections

- User → Web interface: uses
- Web interface → Flask app and API: sends requests
- Flask app and API → Channel lookup: looks up channel
- Channel lookup → YouTube: queries metadata
- Flask app and API → Extraction queue: enqueues channels
- Extraction queue → Queue worker: dispatches jobs
- Queue worker → Channel metadata: resolves channel
- Channel metadata → YouTube: fetches listing
- Queue worker → Video listing: processes videos
- Video listing → YouTube: fetches video data
- Queue worker → Comment extraction: extracts comments
- Comment extraction → YouTube: requests comments
- Queue worker → Channel records: writes metadata
- Queue worker → Video comment records: saves per video
- Flask app and API → Progress and cancellation: reports or stops
- Flask app and API → Channel statistics: serves statistics
- Channel statistics → Channel records: reads metadata
- Channel statistics → Video comment records: counts comments
- Flask app and API → Channel and video detail: serves details
- Channel and video detail → Channel records: reads metadata
- Channel and video detail → Video comment records: reads videos
- Web interface → Insight visualizations: renders insights
- Data loading → Text preprocessing: planned stage
- Text preprocessing → Vectorization: planned stage
- Vectorization → Topic algorithms: planned stage
- Topic algorithms → Dimensionality reduction: planned stage