# haoshuai999/news-summarization architecture

> Architecture diagram of the GitHub repository haoshuai999/news-summarization, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/haoshuai999/news-summarization
- Repository: https://github.com/haoshuai999/news-summarization

## Overview

##### Main Components
a. Data Ingestion & Preprocessing:
- Dataset ingestion of the CNN/Daily Mail corpus.
- Data cleaning, tokenization, and any filtering steps executed in the "Final_project_preprocessing.ipynb".
b. Embedding Generation:
- That embeddings are then fed into the model.
c. Model Architecture (Bidirectional LSTM):
d. Evaluation & Outputs:
- An evaluation component where metrics such as Rough-1 score (0.351) are computed.
- Production of summaries that are extracted from the articles.

##### Relationships and Interactions
- From raw news articles input → Preprocessing module.
- Preprocessing module → Embedding Generation component.
- Embeddings → BiLSTM summarization model.
- BiLSTM model → Evaluation & output of summarized content.

##### Architectural Patterns & Design Principles
- The project follows a modular structure. Each of the notebooks represents a clearly separated phase of the processing pipeline.
- The pipeline can be seen as a chain of data transformations where each module has a single responsibility: input preparation, feature extraction (embeddings), and modeling.
- Data flow orchestration, illustrating how each pipeline stage depends on the previous stage.

##### Technologies, Frameworks, or Libraries
- Mention the use of Jupyter Notebook for experimentation and documentation.
- Although not explicitly listed, the bidirectional LSTM suggests the use of deep learning libraries such as TensorFlow or PyTorch.
- Data preprocessing might involve libraries like pandas, NLTK, or spaCy.

## Diagram (Mermaid)

```mermaid
flowchart TD
    A("CNN/Daily Mail Corpus"):::input
    B("Data Ingestion & Preprocessing"):::process
    C("Embedding Generation"):::process
    D("BiLSTM Modeling & Evaluation"):::process
    E("Summaries & Evaluation [Rough-1:0.351]"):::result

    A -->|"ingests"| B
    B -->|"prepares"| C
    C -->|"feeds embeddings to"| D
    D -->|"outputs"| E
    E -->|"feedback"| D

    click B "https://github.com/haoshuai999/news-summarization/blob/master/Final_project_preprocessing.ipynb"
    click C "https://github.com/haoshuai999/news-summarization/blob/master/Final_project_generate_embeddings.ipynb"
    click D "https://github.com/haoshuai999/news-summarization/blob/master/Final_project_biLSTM_final.ipynb"

    classDef input fill:#fde2e2,stroke:#333,stroke-width:2px;
    classDef process fill:#cce5ff,stroke:#333,stroke-width:2px;
    classDef result fill:#e6ffe6,stroke:#333,stroke-width:2px;
```
