# jung-akim/netflix_app architecture

> Architecture diagram of the GitHub repository jung-akim/netflix_app, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jung-akim/netflix_app
- Repository: https://github.com/jung-akim/netflix_app

## Overview

##### Project type and purpose
- Type: Data-science topic-modeling & classification pipeline with a lightweight web front-end.

##### File structure
- Data directories
- raw_data/ – original CSV/TSV and pickled DataFrames
- preprocessed_data/ – lemmatized corpus, TF-IDF, dictionaries, term-doc matrices
- Model artifacts
- coherence_scores/ – pickled coherence results from grid search
- lda_mallet_model/ – pickled Mallet LDA models
- prediction/ – pickled prediction outputs (LDA & BERT)
- Visualizations
- mallet_lda_vis/ and std_lda_vis/ – pyLDAvis HTML exports
- Code & notebooks
- .ipynb (grid_search, LDA/NMF/LSA, topic_labelling, classification, BERT)
- utils.py – NLP pipeline and helper functions
- predictor_api.py & predictor_app.py – Flask back-end
- templates/predictor.html – simple HTML UI
- Others
- __pycache__ (build artifacts), .ipynb_checkpoints

##### README for additional insights
- Pipeline: Data ingestion → preprocessing → topic modeling → human labeling → BERT training & comparison → 2D visualizations → Flask deployment.
- Tech stack: Python, JupyterLab, gensim, spaCy, NLTK, pyLDAvis, ktrain/transformers, sklearn, umap, wordcloud, Flask.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Source
    AWS_S3["AWS S3 (Amazon Reviews)"]:::external
    RawData[(raw_data/)]:::data
    AWS_S3 -->|"CSV/TSV"| RawData
    click RawData "https://github.com/jung-akim/netflix_app/tree/master/raw_data/"

    %% Preprocessing
    Preprocess["Preprocessing Module\nutils.py\nJupyter Notebooks"]:::proc
    RawData -->|"Pickle"| Preprocess
    click Preprocess "https://github.com/jung-akim/netflix_app/blob/master/utils.py"

    PreData[(preprocessed_data/)]:::data
    Preprocess -->|"PKL, TF-IDF"| PreData
    click PreData "https://github.com/jung-akim/netflix_app/tree/master/preprocessed_data/"

    %% Modeling Module
    subgraph "Modeling Module" 
        direction TB
        %% Grid Search
        GS["Grid Search\n(grid_search.ipynb)"]:::proc
        GS -->|"Pickles"| coherence
        click GS "https://github.com/jung-akim/netflix_app/blob/master/grid_search.ipynb"

        coherence[(coherence_scores/)]:::artifact
        click coherence "https://github.com/jung-akim/netflix_app/tree/master/coherence_scores/"

        %% Topic Modeling
        subgraph "Topic Modeling"
            direction TB
            TM_Mallet["LDA (Mallet)\nlda_mallet_model/"]:::proc
            click TM_Mallet "https://github.com/jung-akim/netflix_app/tree/master/lda_mallet_model/"

            LDA_TM["Standard LDA\nLDA_topic_modeling.ipynb"]:::proc
            click LDA_TM "https://github.com/jung-akim/netflix_app/blob/master/LDA_topic_modeling.ipynb"

            NMF_LSA["NMF & LSA\nNMF_LSA_netflix.ipynb"]:::proc
            click NMF_LSA "https://github.com/jung-akim/netflix_app/blob/master/NMF_LSA_netflix.ipynb"
        end

        coherence --> TM_Mallet
        PreData --> TM_Mallet
        PreData --> LDA_TM
        PreData --> NMF_LSA

        %% Topic Labelling
        Labelling["Topic Labelling\nLDA_topic_labelling.ipynb"]:::proc
        click Labelling "https://github.com/jung-akim/netflix_app/blob/master/LDA_topic_labelling.ipynb"
        TM_Mallet --> Labelling
        LDA_TM --> Labelling
        NMF_LSA --> Labelling

        %% Classification
        subgraph "Classification"
            direction TB
            CLA["LDA Classification\nLDA_classification.ipynb"]:::proc
            click CLA "https://github.com/jung-akim/netflix_app/blob/master/LDA_classification.ipynb"
            CLB["BERT Classification\nBERT.ipynb"]:::proc
            click CLB "https://github.com/jung-akim/netflix_app/blob/master/BERT.ipynb"
        end
        Labelling --> CLA
        Labelling --> CLB

        %% Prediction Outputs
        prediction[(prediction/)]:::artifact
        click prediction "https://github.com/jung-akim/netflix_app/tree/master/prediction/"
        CLA --> prediction
        CLB --> prediction
    end

    %% Visualization Module
    subgraph "Visualization Module"
        direction TB
        vis1[(mallet_lda_vis/)]:::artifact
        click vis1 "https://github.com/jung-akim/netflix_app/tree/master/mallet_lda_vis/"
        vis2[(std_lda_vis/)]:::artifact
        click vis2 "https://github.com/jung-akim/netflix_app/tree/master/std_lda_vis/"
    end
    TM_Mallet -->|"HTML"| vis1
    LDA_TM -->|"HTML"| vis2

    %% Service Layer
    subgraph "Service Layer"
        direction TB
        API["Flask API\npredictor_api.py"]:::service
        click API "https://github.com/jung-akim/netflix_app/blob/master/predictor_api.py"
        UI["Flask UI\npredictor_app.py\ntemplates/predictor.html"]:::service
        click UI "https://github.com/jung-akim/netflix_app/blob/master/predictor_app.py"
        click UI "https://github.com/jung-akim/netflix_app/blob/master/templates/predictor.html"
    end
    prediction -->|"Load Pickles"| API
    UI <-->|"JSON"| API

    %% User
    User["User Browser"]:::external
    User -->|"HTTP"| UI

    %% Styles
    classDef data fill:#ADD8E6,stroke:#000;
    classDef proc fill:#90EE90,stroke:#000;
    classDef artifact fill:#FFA500,stroke:#000;
    classDef service fill:#DDA0DD,stroke:#000;
    classDef external fill:#CCCCCC,stroke:#000;
```
