# harshilpradhan59/video-segmentation-transcript-summarization architecture

> Architecture diagram of the GitHub repository harshilpradhan59/video-segmentation-transcript-summarization, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/harshilpradhan59/video-segmentation-transcript-summarization
- Repository: https://github.com/harshilpradhan59/video-segmentation-transcript-summarization

## Overview

##### Project type and purpose
- This is a machine-learning summarization pipeline for **video lecture transcripts**, implemented in Python and Jupyter.
- It fine-tunes three transformer models (T5, Pegasus, BART), builds an **Ensemble** of their outputs, and evaluates results using ROUGE.

##### File structure
- Top-level “Datasets/” directory holds VT-SSum data.
- “Bart Final.py”, “Pegasus Final.py” and “T5 base Final.ipynb” implement per-model training/fine-tuning.
- “Rouge test.py” runs evaluation scripts.
- “main.py” likely orchestrates the end-to-end pipeline (data loading ➝ training ➝ ensemble ➝ evaluation).
- Reports and ZIP archives (243IT002_…zip, Transcript_Summarizaation_Reportt.pdf) are artifacts or documentation, not runtime components.

##### README for architecture hints
- Methodology flowchart (external link) outlines: data cleaning/segmentation, tokenization, model fine-tuning, ensemble, evaluation.
- Describes use of GPU resources (NVIDIA T4 ×2, P100) and HuggingFace-style transformer architectures.

a. Main components
- Data Ingestion & Preprocessing
- Model Fine-tuning: T5, Pegasus, BART (three parallel pipelines)
- Ensemble Aggregator
- Evaluation (ROUGE)
- Orchestration & Scripts (main.py)
- Storage (raw data, preprocessed data, model checkpoints, evaluation results)
- Compute Resources (GPU cluster)

b. Relationships & interactions
1. Raw data flows from “Datasets/” into a Preprocessor.
2. Preprocessed data is fed in parallel to three training modules (T5, Pegasus, BART).
3. Each training module writes a model checkpoint to Storage.
4. The Ensemble Aggregator reads predictions from each model, merges via averaging/voting, and writes ensemble outputs.
5. Evaluation component reads ensemble outputs (and optionally individual model outputs), computes ROUGE scores, and stores metrics.
6. main.py orchestrates the entire workflow: triggers preprocessing ➝ model training ➝ ensemble ➝ evaluation.

c. Architectural patterns & design principles
- **Modularity**: Each model training pipeline is its own module/script.
- **Data–Compute Separation**: Data storage (Datasets/, artifacts/) is distinct from compute (GPU).
- **Parallelism**: Three transformer pipelines run concurrently.
- **Reproducibility**: Checkpoints and evaluation metrics are versioned in storage.

d. Key technologies & frameworks
- Python scripts + Jupyter Notebook
- HuggingFace Transformers (T5, Pegasus, BART) over PyTorch or TensorFlow
- Pandas/Numpy for preprocessing
- ROUGE metric implementations (Rouge test.py)
- GPU resources (NVIDIA T4s and P100)

##### Detail
- File names or scripts (e.g., “T5 base Final.ipynb” under T5 Trainer).
- Architectural elements so that reviewers instantly understand data volumes, parallelism, and model orchestration.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Stores
    RawData["Raw Lecture Transcripts"]:::datastore
    CP1["T5 Checkpoint"]:::datastore
    CP2["Pegasus Checkpoint"]:::datastore
    CP3["BART Checkpoint"]:::datastore
    Pred1["T5 Predictions"]:::datastore
    Pred2["Pegasus Predictions"]:::datastore
    Pred3["BART Predictions"]:::datastore
    EnsembleOutputs["Ensemble Outputs"]:::datastore
    Metrics["ROUGE Scores"]:::datastore

    %% Compute Modules
    Preprocessor["Preprocessor\n(Cleaning, Tokenization)"]:::compute
    T5["T5 Trainer"]:::compute
    Pegasus["Pegasus Trainer"]:::compute
    BART["BART Trainer"]:::compute
    Ensemble["Ensemble Aggregator"]:::ensemble
    Evaluator["Evaluator"]:::eval
    Orchestrator["Orchestrator\n(main.py)"]:::orchestration

    %% GPU Cluster
    subgraph "GPU Cluster"
        GPU1[/"NVIDIA T4 x2, P100"/]:::compute
    end

    %% Data Flow
    RawData -->|"Raw Data"| Preprocessor
    Preprocessor -->|"Tokenized Inputs"| T5
    Preprocessor -->|"Tokenized Inputs"| Pegasus
    Preprocessor -->|"Tokenized Inputs"| BART

    GPU1 --> T5
    GPU1 --> Pegasus
    GPU1 --> BART

    T5 -->|"Model Checkpoint"| CP1
    Pegasus -->|"Model Checkpoint"| CP2
    BART -->|"Model Checkpoint"| CP3

    T5 -->|"Predictions"| Pred1
    Pegasus -->|"Predictions"| Pred2
    BART -->|"Predictions"| Pred3

    Pred1 -->|"Inputs"| Ensemble
    Pred2 -->|"Inputs"| Ensemble
    Pred3 -->|"Inputs"| Ensemble

    Ensemble -->|"Ensemble Predictions"| EnsembleOutputs

    EnsembleOutputs -->|"Outputs"| Evaluator
    Pred1 -->|"Optional Inputs"| Evaluator
    Pred2 -->|"Optional Inputs"| Evaluator
    Pred3 -->|"Optional Inputs"| Evaluator

    Evaluator -->|"ROUGE Scores"| Metrics

    %% Orchestration
    Orchestrator --> Preprocessor
    Orchestrator --> T5
    Orchestrator --> Pegasus
    Orchestrator --> BART
    Orchestrator --> Ensemble
    Orchestrator --> Evaluator

    %% Click Events
    click Preprocessor "https://github.com/harshilpradhan59/video-segmentation-transcript-summarization/tree/main/Datasets/"
    click T5 "https://github.com/harshilpradhan59/video-segmentation-transcript-summarization/blob/main/T5 base Final.ipynb"
    click Pegasus "https://github.com/harshilpradhan59/video-segmentation-transcript-summarization/blob/main/Pegasus Final.py"
    click BART "https://github.com/harshilpradhan59/video-segmentation-transcript-summarization/blob/main/Bart Final.py"
    click Orchestrator "https://github.com/harshilpradhan59/video-segmentation-transcript-summarization/blob/main/main.py"
    click Evaluator "https://github.com/harshilpradhan59/video-segmentation-transcript-summarization/blob/main/Rouge test.py"

    %% Styles
    classDef datastore fill:#BBDEFB,stroke:#1E88E5;
    classDef compute fill:#C8E6C9,stroke:#43A047;
    classDef ensemble fill:#FFE0B2,stroke:#FB8C00;
    classDef eval fill:#E1BEE7,stroke:#8E24AA;
    classDef orchestration fill:#CFD8DC,stroke:#546E7A;
```
