How sarthakg004/fraud_detection works
Project Type and Purpose
• This is a data-science / machine-learning pipeline project (“Fraud-Detection”) for semi-supervised fraud detection on banking transactions.
• It uses DVC for data and pipeline versioning, MLflow for experiment tracking, and a standard Python project structure under src/.
File Structure
• Top-level:
• dvc.yaml, dvc.lock, .dvc/: pipeline configuration and cache
• data/raw/, data/processed/: versioned datasets
• src/: code modules
• notebooks/: exploratory and prototyping analyses
• mlruns/: MLflow experiment artifacts
• models/: final model artifacts (via DVC or manual commits)
• docs/: Sphinx documentation
• reports/: figures and generated reports
• Makefile, tox.ini, requirements.txt: build, test, and dependency management
• src/ substructure:
• src/data/preprocessing.py
• src/features/feature_engineering.py
• src/models/train_model.py, predict_model.py
• Presence of DVC and MLflow indicates an orchestrated pipeline pattern.
README for Additional Insights
• Confirms purpose: “Fraud detection model for bank transactions”
• Semi-supervised learning suggests an unlabeled data ingestion step and label inference stage.
a. Data Layer
• External Data Source → data/raw/ (DVC tracked)
• data/raw/ → Preprocessing (src/data/preprocessing.py) → data/processed/
b. Feature Engineering
• data/processed/ → Feature Engineering (src/features/feature_engineering.py) → feature store or intermediate artifacts
c. Model Training & Evaluation
• Features → train_model.py → MLflow (mlruns/) logs parameters, metrics, artifacts
• DVC stage in dvc.yaml for train
d. Model Registry & Artifacts
• MLflow model registry or local folder under models/
• Conda/python_env.yaml, requirements.txt captured in MLflow artifact store
e. Prediction Service
• predict_model.py module (could be wrapped behind API or batch job)
• Model → inference on new raw data
f. Experiment Tracking & Orchestration
• DVC orchestrator (dvc.yaml) governs pipeline stages: preprocess → featurize → train → evaluate → deploy
• MLflow for run tracking, logging, and tag management
g. Documentation & Reporting
• docs/ for Sphinx; reports/ for generated figures and write-ups
Architectural Patterns & Design Principles
• Modular pipeline stages, each in its own script and DVC stage
• Separation of concerns: data ingestion, feature engineering, training, inference
• Single source of truth for versioned data (DVC) and metrics/artifacts (MLflow)
• Reproducibility: environment definition in conda/python_env.yaml, pinned dependencies in requirements.txt