# linhnvc/vietnamese-text-summarization architecture

> Architecture diagram of the GitHub repository linhnvc/vietnamese-text-summarization, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/linhnvc/vietnamese-text-summarization
- Repository: https://github.com/linhnvc/vietnamese-text-summarization

## Overview

##### Project type and purpose
- This is a data-science/NLP pipeline project (not a full-stack app).
- Purpose: Automatically generate Vietnamese text summaries by
1) vectorizing sentences via Word2Vec
2) clustering those vectors with K-Means
3) extracting representative sentences as the summary.

##### File structure
- Top-level:
- README.md (project overview)
- docs/ (architecture and algorithm write-ups: NLP.docx, NLP.pptx)
- sources/ (core pipeline code and raw data)
- In sources/:
- Data/ (raw input texts)
- get_data.ipynb (data ingestion & cleaning)
- train_w2v.ipynb (training Word2Vec model)
- using_KMeans.ipynb (clustering sentence embeddings)
- text_summarization.ipynb (assembling the full pipeline)
- hierarchical.ipynb (optional hierarchical clustering experiments)

##### README for additional insights
- Confirms Word2Vec for embeddings and K-Means for clustering.
- No deployment, GUI, or external API layers—purely an offline pipeline.

a. Main components
1. Data Ingestion & Preprocessing
2. Embedding Generation (Word2Vec)
3. Clustering Engine (K-Means)
4. Summary Extraction
5. (Auxiliary) Model Training & Evaluation
6. Documentation & Reports (docs folder)
b. Relationships & interactions
- Raw text → Preprocessing → Tokenized sentences
→ Word2Vec model → Sentence vectors
→ K-Means → Cluster centroids
→ Choose representative sentences → Final summary
c. Architectural patterns/design principles
- Linear data‐processing pipeline (ETL style)
- Modular notebooks for each stage (separation of concerns)
- Reusable models (Word2Vec training vs. inference)
d. Key technologies
- Python / Jupyter Notebooks
- Gensim (Word2Vec)
- scikit-learn (K-Means, optional hierarchical clustering)
- Pandas / NumPy (data manipulation)
- Optionally matplotlib/seaborn for result visualization

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Data Layer
    subgraph "Data Layer"
        RawData[(Raw Data Storage)]:::data
        Preproc["Data Ingestion & Preprocessing"]:::processing
    end

    %% Processing Layer
    subgraph "Processing Layer"
        EmbTraining["Embedding Generation (Word2Vec Training)"]:::processing
        Clustering["Clustering Engine (K-Means Inference)"]:::processing
        HierExp["Hierarchical Clustering Experiment"]:::processing
    end

    %% Model Layer
    subgraph "Model & Inference"
        Summary["Summary Extraction (Pipeline Assembly)"]:::model
        Final["Final Summary"]:::model
    end

    %% Documentation & Reports
    subgraph "Documentation & Reports"
        Readme["README.md"]:::docs
        NLPdoc["NLP.docx"]:::docs
        NLPPPT["NLP.pptx"]:::docs
    end

    %% Data Flow
    RawData -->|"raw texts"| Preproc
    Preproc -->|"tokenized sentences"| EmbTraining
    EmbTraining -->|"sentence embeddings"| Clustering
    EmbTraining -.->|"embeddings"| HierExp
    Clustering -->|"cluster centroids"| Summary
    HierExp -.->|"experimental clusters"| Summary
    Summary -->|"assembled summary"| Final

    %% Click Events
    click Preproc "https://github.com/linhnvc/vietnamese-text-summarization/blob/master/sources/get_data.ipynb"
    click RawData "https://github.com/linhnvc/vietnamese-text-summarization/tree/master/sources/Data/"
    click EmbTraining "https://github.com/linhnvc/vietnamese-text-summarization/blob/master/sources/train_w2v.ipynb"
    click Clustering "https://github.com/linhnvc/vietnamese-text-summarization/blob/master/sources/using_KMeans.ipynb"
    click Summary "https://github.com/linhnvc/vietnamese-text-summarization/blob/master/sources/text_summarization.ipynb"
    click HierExp "https://github.com/linhnvc/vietnamese-text-summarization/blob/master/sources/hierarchical.ipynb"
    click Readme "https://github.com/linhnvc/vietnamese-text-summarization/blob/master/README.md"
    click NLPdoc "https://github.com/linhnvc/vietnamese-text-summarization/blob/master/docs/NLP.docx"
    click NLPPPT "https://github.com/linhnvc/vietnamese-text-summarization/blob/master/docs/NLP.pptx"

    %% Style Definitions
    classDef data fill:#3498db,stroke:#2980b9,color:#fff;
    classDef processing fill:#f39c12,stroke:#e67e22,color:#fff;
    classDef model fill:#2ecc71,stroke:#27ae60,color:#fff;
    classDef docs fill:#bdc3c7,stroke:#95a5a6,color:#2c3e50;
```
