How tpenha05/regulaai_back works
Project type and purpose
• From the file tree and README (“RegulaAI_back”), this is a Python-based machine-learning backend. It uses DVC for data & model versioning, contains an experimental notebook, artifacts (cleaned data and a trained XGBoost pipeline), plus scripts (main.py, prod_model.py) that likely wire up model training and inference.
File structure
• Top-level DVC directory (.dvc/, data.dvc, models.dvc) ⇒ data and model version control is core to the architecture.
• artifacts/ ⇒ stores processed data (df_clean3.parquet) and serialized model (fraud_xgb_pipeline.pkl).
• exp_analysis.ipynb ⇒ interactive notebook for EDA & prototype modeling.
• main.py ⇒ main driver (could define CLI or API endpoints).
• prod_model.py ⇒ inference wrapper around the trained pipeline for production use.
README
• Minimal content, so the file tree is the primary source. The name implies a backend service (RegulaAI_back) focused on regulation or fraud detection.
a. Main components
• Data Versioning (DVC)
• Raw Data Store (e.g., local files or remote storage referenced by DVC)
• Data Processing Module (notebook + code to produce df_clean3.parquet)
• Model Training Module (XGBoost pipeline invoked by main.py or notebook)
• Model Artifact Store (artifacts/fraud_xgb_pipeline.pkl under DVC)
• Production Inference Module (prod_model.py)
• Serving Layer / API (main.py – could be a Flask/FastAPI app)
• External Clients (downstream services or UIs calling the API)
b. Relationships & interactions
1. DVC orchestrates pipelines: pulls raw data ⇒ hands off to Data Processing ⇒ outputs clean dataset ⇒ feeds Model Training ⇒ outputs model artifact ⇒ tags & pushes via DVC.
2. Production path: Serving Layer loads fraud_xgb_pipeline.pkl from artifact store ⇒ prod_model.py runs inference on incoming requests ⇒ returns predictions to External Clients.
c. Architectural patterns / design principles
• MLOps pipeline separation: version control for data & models (DVC).
• Clear separation of experimentation (exp_analysis.ipynb) vs. production code (prod_model.py).
• Modularization: data ingestion, processing, training, and serving each live in their own component.
d. Key technologies
• DVC (data & model version control)
• Python (ML code, serving scripts)
• XGBoost (modeling library)
• Jupyter (exploratory notebook)
• Parquet (cleaned data format)
• (Implicit) Flask/FastAPI or CLI in main.py / prod_model.py
Final recommendations
• DVC config as a side-bar process that versions both data and model outputs.
• Callouts for key files (fraud_xgb_pipeline.pkl, df_clean3.parquet, prod_model.py).
• Don’t over-complicate: keep the pipeline as discrete stages for maximum clarity.