How gillopy/deployment_xgboost_heart_disease_uci works
Main Components
• Data Layer:
• Input Data Folder: Represented by the "data/" directory. This is the starting point where raw datasets (e.g., heart.csv) are stored.
• Data Modules: Under "src/data/", include:
▪ data_loader.py – Responsible for loading the dataset.
▪ data_processor.py – Handles any preprocessing and transformation tasks.
▪ data_splitter.py – Splits the data into training and testing sets.
• Model Layer:
• Training and Evaluation Components: Under "src/model/", incorporate:
▪ trainer.py – Handles the training of the model using XGBoost.
▪ evaluator.py – Evaluates the model performance (e.g., calculates accuracy).
▪ saver.py – Saves the trained models in the "models/" folder with a timestamp.
• Orchestration:
• Main Pipeline: "src/main.py" coordinates all these processes, linking data ingestion, model training, evaluation, and saving.
Relationships and Interactions
• Next, indicate that the processed and split data flows into the Trainer component, which invokes XGBoost training.
• Finally, connect the Evaluator to Saver, reflecting that once the model is evaluated, it is saved in the "models/" directory.
Architectural Patterns and Design Principles
• Separation of Concerns: Each module (data handling, training, evaluation) is in its own directory under "src/". This structure demonstrates decoupling and organization.
• Pipeline Architecture: The system works as a sequential training pipeline from data ingestion through model training and saving.
• Extensibility: The modular design makes it easier to update or replace components, such as switching model parameters in trainer.py or updating preprocessing in data_processor.py.
Relevant Technologies and Frameworks
• Python: The core language, with version 3.10+.
• Poetry: Used for dependency management (configured via pyproject.toml).
• XGBoost: The key library used for model training.
• Pandas: Utilized in the data loading and processing steps.
• Scikit-learn: Used for data splitting and possibly some evaluation metrics.
• Joblib: For saving and loading trained models.
Detailed Elements
• Use clear, descriptive labels for each component (e.g., Data Loader, Data Processor, Trainer, Evaluator, Saver).
• E.g., from Data Loader to Processor, then to Splitter, and subsequently into Training.
• Data-related modules (perhaps in blue/rounded rectangles),
• Model-related modules (maybe in green/rectangles),
Final Touches
• Adding any infrastructural components not explicitly shown in the file tree if they might be assumed (e.g., version control using Git, Docker for deployment as seen by the Dockerfile).
• Although the current focus is on model training and deployment, note that the modular design permits scalability, so represent how each component might be extended separately.