How francesco-s/synthetic-email-generation-and-classification works
Project type and purpose
• This is a standalone machine-learning pipeline for synthetic email generation and classification, orchestrated via a single entrypoint (main.py) and optionally containerized with Docker/Docker Compose.
• It covers two major concerns:
• Synthetic data generation (via a GPT-4-based generator)
• Model training & evaluation (Logistic Regression, SVM, Gradient Boosting, Random Forest)
File structure
• Top-level files:
• main.py: coordinator that loads data (or generates it), invokes preprocessing, training, evaluation.
• Dockerfile & docker-compose.yml: containerization and orchestration.
• requirements.txt, example.env, README.md, report.pdf.
• Directories:
• data/: raw or generated CSV files (synthetic_emails_*.csv).
• generator/: synthetic_generator.py for invoking OpenAI GPT API to create synthetic emails.
• classifier/:
– data_processing.py: loading & preprocessing (tokenization, feature extraction).
– modeling.py: pipeline setup (vectorizer + classifier).
– model_training.py: cross-validation loops.
– evaluation.py: metrics (accuracy, precision, recall, F1).
– model/random_forest.pickle: example serialized model.
• models/: Pydantic schema (email_model.py) for validating generator output.
• prompts/: templates & examples to drive the GPT-4 generator.
README for additional insights
• Confirms the two-phase workflow: data ingestion/generation → model training & evaluation.
• Indicates environment variables (OpenAI key) and container workflows.
• No external database; all data is file-based.
Detail
• Submodule in classifier/ as separate components if desired (data_processing, modeling, training, evaluation).
• Configuration flow (config file + .env → modules).
• Docker Compose network linking the container to the host’s file system (data/, config/).