# francesco-s/synthetic-email-generation-and-classification architecture

> Architecture diagram of the GitHub repository francesco-s/synthetic-email-generation-and-classification, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/francesco-s/synthetic-email-generation-and-classification
- Repository: https://github.com/francesco-s/synthetic-email-generation-and-classification

## Overview

##### Project type and purpose
- This is a standalone machine-learning pipeline for synthetic email generation and classification, orchestrated via a single entrypoint (`main.py`) and optionally containerized with Docker/Docker Compose.
- It covers two major concerns:
- Synthetic data generation (via a GPT-4-based generator)
- Model training & evaluation (Logistic Regression, SVM, Gradient Boosting, Random Forest)

##### File structure
- Top-level files:
- `main.py`: coordinator that loads data (or generates it), invokes preprocessing, training, evaluation.
- `Dockerfile` & `docker-compose.yml`: containerization and orchestration.
- `requirements.txt`, `example.env`, `README.md`, `report.pdf`.
- Directories:
- `data/`: raw or generated CSV files (synthetic_emails_*.csv).
- `generator/`: `synthetic_generator.py` for invoking OpenAI GPT API to create synthetic emails.
- `classifier/`:
– `data_processing.py`: loading & preprocessing (tokenization, feature extraction).
– `modeling.py`: pipeline setup (vectorizer + classifier).
– `model_training.py`: cross-validation loops.
– `evaluation.py`: metrics (accuracy, precision, recall, F1).
– `model/random_forest.pickle`: example serialized model.
- `models/`: Pydantic schema (`email_model.py`) for validating generator output.
- `prompts/`: templates & examples to drive the GPT-4 generator.

##### README for additional insights
- Confirms the two-phase workflow: data ingestion/generation → model training & evaluation.
- Indicates environment variables (OpenAI key) and container workflows.
- No external database; all data is file-based.

##### Detail
- Submodule in `classifier/` as separate components if desired (data_processing, modeling, training, evaluation).
- Configuration flow (config file + `.env` → modules).
- Docker Compose network linking the container to the host’s file system (`data/`, `config/`).

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% Orchestration Layer
    subgraph "Containerization & Orchestration"
        direction TB
        DockerCompose["docker-compose.yml"]:::orchestration
        DockerfileNode["Dockerfile"]:::orchestration
    end

    DockerCompose -->|runs| Main["main.py"]:::orchestration
    DockerfileNode -->|builds| Main

    %% Configuration Layer
    ConfigFile["config/config.py"]:::config
    EnvFile["example.env"]:::config
    ConfigFile -->|loads| Main
    EnvFile -->|provides| Main

    %% Main Orchestration
    Main -->|checks| CheckData["Data Directory Check"]:::data

    %% Data Layer
    subgraph "Data Layer"
        direction TB
        DataStorage["CSV Files\n(data/)"]:::data
    end

    CheckData -->|exists| DataStorage
    CheckData -->|empty| Generator["Synthetic Generator"]:::data

    %% Generator Branch
    subgraph "Synthetic Data Generation"
        direction TB
        Prompts["Prompt Templates & Examples\n(prompts/)"]:::data
        OpenAI["OpenAI API"]:::external
        Pydantic["Pydantic Schema\n(models/email_model.py)"]:::config
    end

    Prompts --> Generator
    Generator --> OpenAI
    OpenAI --> Generator
    Generator --> Pydantic
    Pydantic --> Generator
    Generator --> DataStorage

    %% Processing & Training Layer
    DataStorage --> DataProcessing["Data Processing Module\n(classifier/data_processing.py)"]:::processing
    DataProcessing --> ModelDef["Model Pipeline Definition\n(classifier/modeling.py)"]:::processing
    ModelDef --> Training["Model Training & Cross-Validation\n(classifier/model_training.py)"]:::processing
    Training --> Evaluation["Evaluation & Metrics Computation\n(classifier/evaluation.py)"]:::processing
    Evaluation --> Persistence["Model Persistence\n(classifier/model/random_forest.pickle)"]:::processing

    %% Click Events
    click DataStorage "https://github.com/francesco-s/synthetic-email-generation-and-classification/tree/main/data/"
    click Generator "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/generator/synthetic_generator.py"
    click Prompts "https://github.com/francesco-s/synthetic-email-generation-and-classification/tree/main/prompts/"
    click ConfigFile "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/config/config.py"
    click EnvFile "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/example.env"
    click DataProcessing "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/classifier/data_processing.py"
    click ModelDef "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/classifier/modeling.py"
    click Training "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/classifier/model_training.py"
    click Evaluation "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/classifier/evaluation.py"
    click Persistence "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/classifier/model/random_forest.pickle"
    click Pydantic "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/models/email_model.py"
    click Main "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/main.py"
    click DockerCompose "https://github.com/francesco-s/synthetic-email-generation-and-classification/blob/main/docker-compose.yml"
    click DockerfileNode "https://github.com/francesco-s/synthetic-email-generation-and-classification/tree/main/Dockerfile"

    %% Styles
    classDef data fill:#cce5ff,stroke:#034694,color:#034694
    classDef processing fill:#ffe5cc,stroke:#c56c00,color:#c56c00
    classDef external fill:#ccffcc,stroke:#2e7d32,color:#2e7d32
    classDef orchestration fill:#e0e0e0,stroke:#5f5f5f,color:#5f5f5f
    classDef config fill:#f0f0f0,stroke:#6d6d6d,color:#6d6d6d
```
