How daya-1-sgr/online_payment_fraud_prediction works
Main Components
• Data Source & Artifacts:
• Raw dataset input (from Kaggle) and stored artifact files (CSV data, pickled preprocessing objects, trained models, etc.) located in the artifacts directory.
• Data Ingestion Component:
• Located in src/components/data_ingestion.py. This component handles loading and providing data from CSV files and possibly other sources.
• Data Transformation Component:
• Found in src/components/data_transformation.py. It preprocesses data (handling missing values, encoding categorical variables, scaling etc.). Probability of additional preprocessing pipelines from notebooks (like EDA.ipynb) can be noted as exploratory or validation layers.
• Model Trainer Component:
• Indicated by src/components/model_trainer.py and the catboost_info folder. This part encapsulates model training using various classifiers and records training logs or metrics.
• Prediction & Training Pipelines:
• Under the src/pipeline/ directory with predict_pipeline.py and train_pipeline.py. These orchestrate the overall workflow from data ingestion to prediction, supporting both training mode and real-time predictions.
• Real-time Prediction Interface:
• Provided by streamlit_app.py, which launches a Streamlit web application for interactive fraud detection.
• Logging & Exception Handling:
• With source files such as src/logger.py and src/exception.py to handle logging and error management.
• Supporting Utilities:
• The src/utils.py file, which likely includes helper functions that support the main components.
Relationships and Interactions
• Data starts at the input layer (data from artifacts or via direct ingestion from CSVs).
• It passes through the Data Ingestion component.
• Then flows into Data Transformation where preprocessing is applied.
• After preprocessing, the cleaned data is fed to the Model Trainer for training, with the training logs stored along with model artifacts.
• For real-time predictions, the Prediction Pipeline (predict_pipeline.py) takes incoming data (from the Streamlit interface via streamlit_app.py) and uses the already-trained model (from the artifacts directory) to output predictions.
• Supporting interactions such as logging, exception handling, and utility functions that tie into every stage.
Architectural Patterns
• Separation of Concerns: Each component (ingestion, transformation, training, and prediction) is clearly separated.
• Pipeline Architecture: The use of separate train and predict pipelines under the src/pipeline directory.
• Modular Design: With dedicated modules for components, logging, and utilities.
• Real-time vs. Batch: The design supports both offline batch processing (training) and real-time inference through Streamlit.
Additional Diagrammatic Elements
• Configuration setups like the .devcontainer for environment consistency.
• Notebook files as an auxiliary exploratory component used during EDA and model validation.
Summary of What
o One heading to the "Model Trainer" (with a sub-note on model training/validation using CatBoost and other classification algorithms)
o The other leading to the "Train Pipeline" which integrates with the model trainer.
• A separate flow for the "Prediction Pipeline" which connects from the artifacts storage (where the trained model is stored) to the front-end.
• Supporting modules: "Logger" and "Exception Handling" that interact with various stages.