How harshilpradhan59/video-segmentation-transcript-summarization works
Project type and purpose
• This is a machine-learning summarization pipeline for video lecture transcripts, implemented in Python and Jupyter.
• It fine-tunes three transformer models (T5, Pegasus, BART), builds an Ensemble of their outputs, and evaluates results using ROUGE.
File structure
• Top-level “Datasets/” directory holds VT-SSum data.
• “Bart Final.py”, “Pegasus Final.py” and “T5 base Final.ipynb” implement per-model training/fine-tuning.
• “Rouge test.py” runs evaluation scripts.
• “main.py” likely orchestrates the end-to-end pipeline (data loading ➝ training ➝ ensemble ➝ evaluation).
• Reports and ZIP archives (243IT002_…zip, Transcript_Summarizaation_Reportt.pdf) are artifacts or documentation, not runtime components.
README for architecture hints
• Methodology flowchart (external link) outlines: data cleaning/segmentation, tokenization, model fine-tuning, ensemble, evaluation.
• Describes use of GPU resources (NVIDIA T4 ×2, P100) and HuggingFace-style transformer architectures.
a. Main components
• Data Ingestion & Preprocessing
• Model Fine-tuning: T5, Pegasus, BART (three parallel pipelines)
• Ensemble Aggregator
• Evaluation (ROUGE)
• Orchestration & Scripts (main.py)
• Storage (raw data, preprocessed data, model checkpoints, evaluation results)
• Compute Resources (GPU cluster)
b. Relationships & interactions
1. Raw data flows from “Datasets/” into a Preprocessor.
2. Preprocessed data is fed in parallel to three training modules (T5, Pegasus, BART).
3. Each training module writes a model checkpoint to Storage.
4. The Ensemble Aggregator reads predictions from each model, merges via averaging/voting, and writes ensemble outputs.
5. Evaluation component reads ensemble outputs (and optionally individual model outputs), computes ROUGE scores, and stores metrics.
6. main.py orchestrates the entire workflow: triggers preprocessing ➝ model training ➝ ensemble ➝ evaluation.
c. Architectural patterns & design principles
• Modularity: Each model training pipeline is its own module/script.
• Data–Compute Separation: Data storage (Datasets/, artifacts/) is distinct from compute (GPU).
• Parallelism: Three transformer pipelines run concurrently.
• Reproducibility: Checkpoints and evaluation metrics are versioned in storage.
d. Key technologies & frameworks
• Python scripts + Jupyter Notebook
• HuggingFace Transformers (T5, Pegasus, BART) over PyTorch or TensorFlow
• Pandas/Numpy for preprocessing
• ROUGE metric implementations (Rouge test.py)
• GPU resources (NVIDIA T4s and P100)
Detail
• File names or scripts (e.g., “T5 base Final.ipynb” under T5 Trainer).
• Architectural elements so that reviewers instantly understand data volumes, parallelism, and model orchestration.