How linhnvc/vietnamese-text-summarization works
Project type and purpose
• This is a data-science/NLP pipeline project (not a full-stack app).
• Purpose: Automatically generate Vietnamese text summaries by
1) vectorizing sentences via Word2Vec
2) clustering those vectors with K-Means
3) extracting representative sentences as the summary.
File structure
• Top-level:
• README.md (project overview)
• docs/ (architecture and algorithm write-ups: NLP.docx, NLP.pptx)
• sources/ (core pipeline code and raw data)
• In sources/:
• Data/ (raw input texts)
• get_data.ipynb (data ingestion & cleaning)
• train_w2v.ipynb (training Word2Vec model)
• using_KMeans.ipynb (clustering sentence embeddings)
• text_summarization.ipynb (assembling the full pipeline)
• hierarchical.ipynb (optional hierarchical clustering experiments)
README for additional insights
• Confirms Word2Vec for embeddings and K-Means for clustering.
• No deployment, GUI, or external API layers—purely an offline pipeline.
a. Main components
1. Data Ingestion & Preprocessing
2. Embedding Generation (Word2Vec)
3. Clustering Engine (K-Means)
4. Summary Extraction
5. (Auxiliary) Model Training & Evaluation
6. Documentation & Reports (docs folder)
b. Relationships & interactions
• Raw text → Preprocessing → Tokenized sentences
→ Word2Vec model → Sentence vectors
→ K-Means → Cluster centroids
→ Choose representative sentences → Final summary
c. Architectural patterns/design principles
• Linear data‐processing pipeline (ETL style)
• Modular notebooks for each stage (separation of concerns)
• Reusable models (Word2Vec training vs. inference)
d. Key technologies
• Python / Jupyter Notebooks
• Gensim (Word2Vec)
• scikit-learn (K-Means, optional hierarchical clustering)
• Pandas / NumPy (data manipulation)
• Optionally matplotlib/seaborn for result visualization