# jkcreddy/text-summarizer architecture

> Architecture diagram of the GitHub repository jkcreddy/text-summarizer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-12). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jkcreddy/text-summarizer
- Repository: https://github.com/jkcreddy/text-summarizer

## Overview

##### Project Type and Purpose
- This is an end-to-end machine-learning pipeline and serving application for text summarization.
- It ingests raw data, validates and transforms it, trains and evaluates a model, then exposes a REST endpoint (`app.py`) for on-demand summarization.
- It also includes CI/CD via GitHub Actions to build a Docker image, push it to AWS ECR, and deploy to an EC2 instance.

##### High-Level File Structure Analysis
- Top-level scripts:
- main.py – orchestrates the pipeline (CLI).
- app.py – runs the web service (Flask/FastAPI) for inference.
- Configuration:
- config/config.yaml, params.yaml – hyperparameters, file paths.
- src/textSummarizer/config/configuration.py – loads and provides config objects.
- Pipeline code under `src/textSummarizer`:
- components/ – modular stages (data_ingestion, validation, transformation, trainer, evaluator).
- pipeline/ – stage orchestration scripts (stage_01…05) plus `prediction.py`.
- utils/, logging/, entity/, constants/ – shared helpers, data models, logging setup.
- Artifacts folder:
- artifacts/data_ingestion/… – raw and Hugging Face–formatted dataset files.
- Deployment & CI:
- Dockerfile – containerizes the app.
- .github/workflows/main.yaml – CI/CD pipeline.
- AWS secrets, ECR/EC2 instructions in README.

##### Main Components & Relationships
- Component: Configuration Manager
- Reads config/config.yaml & params.yaml → supplies paths & hyperparameters.
- Component: Data Storage
- Raw data (artifacts/data_ingestion/data.zip) → Hugging Face dataset files under artifacts/data_ingestion/samsum_dataset.
- Component: Pipeline Stages
1. Data Ingestion
2. Data Validation
3. Data Transformation
4. Model Training
5. Model Evaluation
- Each implemented in src/textSummarizer/components and invoked by src/textSummarizer/pipeline/stage_* scripts.
- Component: Model Artifacts
- Trained model files (not explicitly shown, but saved via trainer component).
- Component: Prediction Service
- `prediction.py` provides inference logic.
- `app.py` wraps it in a REST API.
- CI/CD Workflow
1. Developer commits code to GitHub.
2. GitHub Actions builds Docker image (Dockerfile + requirements.txt).
3. Pushes image to AWS ECR.
4. EC2 self-hosted runner pulls image from ECR.
5. Launches the container, exposing the API endpoint.

##### Architectural Patterns & Design Principles
- Modular separation: each pipeline stage is self-contained.
- Configuration-driven: YAML files define behavior without code changes.
- Clear layering: components → pipeline orchestrator → CLI (`main.py`) or service (`app.py`).
- CI/CD automation for reproducible builds and deployments.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Configuration Manager
    ConfigManager["Configuration Manager"]:::compute

    %% Data Storage
    DataStorage["Artifacts Data Storage"]:::data
    DataStorage -->|"contains raw & processed datasets"| ArtifactsFiles

    %% List artifact files
    ArtifactsFiles["samsum_dataset & CSVs"]:::data

    %% Shared Helpers
    SharedHelpers["Shared Helpers & Entities"]:::compute

    %% Pipeline Stages
    subgraph "Pipeline Stages"
        direction TB
        DataIngestion["Data Ingestion"]:::compute
        DataValidation["Data Validation"]:::compute
        DataTransformation["Data Transformation"]:::compute
        ModelTraining["Model Training"]:::compute
        ModelEvaluation["Model Evaluation"]:::compute
    end

    %% Orchestration & Entry
    MainCLI["main.py CLI"]:::compute
    AppService["app.py REST API"]:::compute

    %% Prediction Service
    Prediction["Prediction Service"]:::compute

    %% Model Artifacts
    ModelArtifacts["Model Artifacts"]:::data

    %% CI/CD Pipeline
    subgraph "CI/CD Workflow"
        direction TB
        GitHub["GitHub Repo"]:::ci
        Actions["GitHub Actions"]:::ci
        DockerImage["Docker Image"]:::ci
        ECR["AWS ECR"]:::cloud
        EC2["AWS EC2"]:::cloud
        Container["Running Container"]:::compute
    end

    %% Flows
    ConfigManager -->|reads config| DataIngestion
    ConfigManager -->|provides params| DataValidation
    ConfigManager -->|provides params| DataTransformation
    ConfigManager -->|provides params| ModelTraining
    ConfigManager -->|provides params| ModelEvaluation

    DataStorage -->|reads raw data| DataIngestion
    DataIngestion -->|produces validated data| DataValidation
    DataValidation -->|produces transformed data| DataTransformation
    DataTransformation -->|produces training set| ModelTraining
    ModelTraining -->|saves model| ModelArtifacts
    ModelArtifacts -->|used for evaluation| ModelEvaluation
    ModelEvaluation -->|reports metrics| ModelArtifacts

    ModelArtifacts -->|loads model| Prediction
    ConfigManager -->|provides config| Prediction
    Prediction -->|inference logic| AppService
    MainCLI -->|orchestrates stages| DataIngestion
    AppService -->|HTTP request| Prediction

    %% CI/CD flows
    GitHub -->|push code| Actions
    Actions -->|build image| DockerImage
    DockerImage -->|push image| ECR
    ECR -->|pull image| EC2
    EC2 -->|deploy container| Container
    Container -->|serves API| AppService

    %% Click events
    click ConfigManager "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/config/configuration.py"
    click ConfigManager "https://github.com/jkcreddy/text-summarizer/blob/main/config/config.yaml"
    click ConfigManager "https://github.com/jkcreddy/text-summarizer/blob/main/params.yaml"
    click DataStorage "https://github.com/jkcreddy/text-summarizer/blob/main/artifacts/data_ingestion/data.zip"
    click DataStorage "https://github.com/jkcreddy/text-summarizer/blob/main/artifacts/data_ingestion/samsum-train.csv"
    click DataStorage "https://github.com/jkcreddy/text-summarizer/blob/main/artifacts/data_ingestion/samsum-validation.csv"
    click DataStorage "https://github.com/jkcreddy/text-summarizer/blob/main/artifacts/data_ingestion/samsum-test.csv"
    click DataStorage "https://github.com/jkcreddy/text-summarizer/tree/main/artifacts/data_ingestion/samsum_dataset/"
    click DataIngestion "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/components/data_ingestion.py"
    click DataIngestion "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/pipeline/stage_01_data_ingestion.py"
    click DataValidation "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/components/data_validation.py"
    click DataValidation "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/pipeline/stage_02_data_validation.py"
    click DataTransformation "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/components/data_transformation.py"
    click DataTransformation "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/pipeline/stage_03_data_transformation.py"
    click ModelTraining "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/components/model_trainer.py"
    click ModelTraining "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/pipeline/stage_04_model_trainer.py"
    click ModelEvaluation "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/components/model_evaluation.py"
    click ModelEvaluation "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/pipeline/stage_05_model_evaluation.py"
    click SharedHelpers "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/utils/common.py"
    click SharedHelpers "https://github.com/jkcreddy/text-summarizer/tree/main/src/textSummarizer/logging/"
    click SharedHelpers "https://github.com/jkcreddy/text-summarizer/tree/main/src/textSummarizer/constants/"
    click SharedHelpers "https://github.com/jkcreddy/text-summarizer/tree/main/src/textSummarizer/entity/"
    click MainCLI "https://github.com/jkcreddy/text-summarizer/blob/main/main.py"
    click AppService "https://github.com/jkcreddy/text-summarizer/blob/main/app.py"
    click Prediction "https://github.com/jkcreddy/text-summarizer/blob/main/src/textSummarizer/pipeline/prediction.py"
    click GitHub "https://github.com/jkcreddy/text-summarizer/blob/main/ .github/workflows/main.yaml"
    click GitHub "https://github.com/jkcreddy/text-summarizer/tree/main/Dockerfile"
    click GitHub "https://github.com/jkcreddy/text-summarizer/blob/main/requirements.txt"
    click GitHub "https://github.com/jkcreddy/text-summarizer/blob/main/setup.py"

    %% Styles
    classDef data fill:#AED6F1,stroke:#1F618D;
    classDef compute fill:#A9DFBF,stroke:#145A32;
    classDef ci fill:#F9E79F,stroke:#B9770E;
    classDef cloud fill:#E5E8E8,stroke:#566573;
```
