# faroukelmak/textsummarizer architecture

> Architecture diagram of the GitHub repository faroukelmak/textsummarizer, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-11-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/faroukelmak/textsummarizer
- Repository: https://github.com/faroukelmak/textsummarizer

## Overview

##### Project type and purpose
- This is a machine-learning text summarization pipeline packaged as a Python library/service with both training and prediction capabilities.
- The README mentions workflows for configuration, parameter management, data ingestion, transformation, model training/evaluation, and APIs for training and batch prediction.

##### File structure
- Top-level files:
- app.py, main.py – likely entry points for serving (API) and orchestration
- Dockerfile, requirements.txt, setup.py – containerization, dependency management, packaging
- config/config.yaml and params.yaml – static configuration and hyperparameters
- src/TextSummarizer:
- components/ – individual stages (data_ingestion, data_transformation, model_trainer, model_evaluation)
- pipeline/ – pipelines that stitch together components for each stage and prediction
- config/ – configuration loader and manager
- constants/, entity/, utils/, logging/ – support modules for shared constants, data models, utilities, and logging
- research/ – Jupyter notebooks for exploratory work (data ingest through evaluation)

##### README for additional insights
- Emphasis on a Configuration entity and manager
- A clear 4-stage pipeline (ingestion → transformation → training → evaluation)
- Two pipelines exposed to users: Training Pipeline and Prediction Pipeline
- A lightweight “front end” via API endpoints for training and batch prediction

##### Main components and their interactions
a. Configuration Layer
- config.yaml + params.yaml → Configuration Manager → Config Entity
b. Data Pipeline Components
- Data Ingestion → Data Transformation → Model Trainer → Model Evaluation
- Each component reads from disk or previous artifact, writes output artifacts (e.g., transformed data, trained model, evaluation metrics)
c. Pipeline Orchestrator
- stage_1…stage_4 pipelines invoke components in sequence
- prediction_pipeline loads the trained model artifact and serves predictions
d. API Layer
- app.py/main.py expose REST endpoints:
– /train triggers the Training Pipeline
– /predict (batch) invokes the Prediction Pipeline on input data
e. Support Modules
- utils/common for shared helper functions
- logging to standardize logs across components
- constants and entity to define data schemas and file paths

##### Architectural patterns and design principles
- Layered modular design: clear separation of configuration, components, pipelines, API
- Pipeline pattern: each stage is a self-contained component
- Single monolithic service containerized via Docker
- Use of dependency injection for config to decouple code from static parameters

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External User
    User(["User/API Client"]):::external

    subgraph "Docker Container"
        direction TB

        %% API Layer
        API["API Layer\n(app.py, main.py)"]:::api

        %% Configuration Manager
        ConfigMgr["Configuration Manager"]:::component

        %% Support Modules
        subgraph "Support Modules"
            direction TB
            Constants["Constants Module"]:::support
            Entity["Entity Definitions"]:::support
            Utils["Utilities (common)"]:::support
            Logging["Logging Module"]:::support
        end

        %% Pipelines
        subgraph "Pipelines" 
            direction TB
            TrainingPipe["Training Pipeline"]:::pipeline
            PredictionPipe["Prediction Pipeline"]:::pipeline

            %% Training stages
            subgraph "Training Stages"
                direction TB
                Stage1Pipe["Stage 1: Data Ingestion"]:::pipeline
                Stage2Pipe["Stage 2: Data Transformation"]:::pipeline
                Stage3Pipe["Stage 3: Model Trainer"]:::pipeline
                Stage4Pipe["Stage 4: Model Evaluation"]:::pipeline
            end
        end

        %% Components
        subgraph "Components"
            direction TB
            IngestComp["Data Ingestion Component"]:::component
            TransformComp["Data Transformation Component"]:::component
            TrainerComp["Model Trainer Component"]:::component
            EvalComp["Model Evaluation Component"]:::component
        end

        %% Artifact Storage
        Storage(["Artifact Storage\n(local filesystem)"]):::storage

        %% Connections inside container
        ConfigMgr -->|reads config| Stage1Pipe
        ConfigMgr -->|reads config| Stage2Pipe
        ConfigMgr -->|reads config| Stage3Pipe
        ConfigMgr -->|reads config| Stage4Pipe
        ConfigMgr -->|reads config| PredictionPipe

        Constants --> IngestComp
        Constants --> TransformComp
        Constants --> TrainerComp
        Constants --> EvalComp
        Entity --> IngestComp
        Entity --> TransformComp
        Entity --> TrainerComp
        Entity --> EvalComp
        Utils --> IngestComp
        Utils --> TransformComp
        Utils --> TrainerComp
        Utils --> EvalComp
        Logging --> IngestComp
        Logging --> TransformComp
        Logging --> TrainerComp
        Logging --> EvalComp

        %% Training pipeline flow
        TrainingPipe --> Stage1Pipe
        Stage1Pipe --> IngestComp
        IngestComp -->|outputs raw_data.csv| Stage2Pipe
        Stage2Pipe --> TransformComp
        TransformComp -->|outputs transformed_data.csv| Stage3Pipe
        Stage3Pipe --> TrainerComp
        TrainerComp -->|writes model.pkl| Stage4Pipe
        Stage4Pipe --> EvalComp
        EvalComp -->|writes metrics.json| Storage

        %% Prediction pipeline flow
        PredictionPipe -->|loads model.pkl| Storage
        PredictionPipe -->|predicts batch| Storage

        %% API connections
        API -->|POST /train| TrainingPipe
        API -->|POST /predict| PredictionPipe
    end

    %% External connections
    User --> API

    %% Click Events
    click ConfigMgr "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/config/configuration.py"
    click IngestComp "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/components/data_ingestion.py"
    click Stage1Pipe "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/pipeline/stage_1_data_ingestion_pipeline.py"
    click TransformComp "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/components/data_transformation.py"
    click Stage2Pipe "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/pipeline/stage_2_data_transformation_pipeline.py"
    click TrainerComp "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/components/model_trainer.py"
    click Stage3Pipe "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/pipeline/stage_3_model_trainer_pipeline.py"
    click EvalComp "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/components/model_evaluation.py"
    click Stage4Pipe "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/pipeline/stage_4_model_evaluation_pipeline.py"
    click PredictionPipe "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/pipeline/prediction_pipeline.py"
    click API "https://github.com/faroukelmak/textsummarizer/blob/main/app.py"
    click Constants "https://github.com/faroukelmak/textsummarizer/tree/main/src/TextSummarizer/constants"
    click Entity "https://github.com/faroukelmak/textsummarizer/tree/main/src/TextSummarizer/entity"
    click Utils "https://github.com/faroukelmak/textsummarizer/blob/main/src/TextSummarizer/utils/common.py"
    click Logging "https://github.com/faroukelmak/textsummarizer/tree/main/src/TextSummarizer/logging"

    %% Styles
    classDef component fill:#add8e6,stroke:#333,stroke-width:1px
    classDef pipeline fill:#90ee90,stroke:#333,stroke-width:1px
    classDef storage fill:#f4a460,stroke:#333,stroke-width:1px
    classDef api fill:#87ceeb,stroke:#333,stroke-width:1px
    classDef support fill:#d3d3d3,stroke:#333,stroke-width:1px
    classDef external fill:#ffc0cb,stroke:#333,stroke-width:1px
```
