rohit-dhade/pre-delinquency-intervention-engine

1. Repository split

- This repo is a two-service system: a Python Delinquency_Engine for scoring risk, and a Node.js Intervention_Engine for deciding and sending customer outreach.

- They communicate over HTTP, while both rely on the same PostgreSQL-backed operational data.

2. Delinquency Engine boundary

- The FastAPI service is the ML inference layer.

- Its job is to fetch features, run the XGBoost classifier, generate SHAP-based explanations, and optionally trigger intervention downstream.

3. Prediction data flow

- Customer data originates in PostgreSQL tables such as customers, transactions, and payment_events.

- Daily feature SQL produces engineered features, which are materialized into a Feast online store.

- The FastAPI endpoint /predict/{customer_id} reads from Feast, scores delinquency probability, and returns the prediction plus explanations.

4. Modeling and explainability

- The trained artifacts are packaged directly in the backend: xgb_model_latest.joblib, shap_explainer.joblib, and model_feature_names.joblib.

- SHAP identifies the main drivers of risk, and those top reasons are converted into human-readable text via Mistral-backed explanation generation.

5. Feature store subsystem

- Feast is the feature-serving boundary between raw transactional data and model inference.

- The offline source is a Parquet snapshot under feast_repo/data/, while the online store is a local SQLite DB used for low-latency point lookups by customer_id.

- The Feast repo also defines the entity and feature views that keep training and serving aligned.

6. Feature pipeline and orchestration

- The scripts/ directory contains the data and ML pipeline: synthetic data generation, SQL feature computation, Feast materialization, traffic simulation, and weekly retraining.

- Airflow runs the daily feature pipeline DAG, while weekly_retrain.py handles retraining gates and model hot-reload.

7. Intervention Engine boundary

- The Node.js service is the business-action layer.

- It receives a delinquency signal, applies policy rules, chooses an offer, generates messaging, sends email, and logs the outcome.

8. Intervention decision pipeline

- tierRouter converts probability into risk tiers like stable, watch, moderate, and critical.

- offerEngine maps the tier plus customer context to a relief product.

- channelRouter determines urgency and routing, and messageGenerator produces the customer-facing message.

9. Messaging and delivery

- Intervention emails are generated with Mistral AI, then wrapped in branded HTML and delivered through Nodemailer using Gmail SMTP.

- If AI generation fails, the service falls back to predefined templates, so email delivery is not dependent on the model being available.

10. Shared persistence

- PostgreSQL is the system’s state backbone.

- The Delinquency Engine owns the source and feature tables plus retraining audit logs.

- The Intervention Engine writes to intervention_log and outcome_log, which track outreach actions and customer responses.

11. Service-to-service coupling

- The coupling is intentionally narrow: FastAPI calls the Intervention Engine only when the delinquency probability crosses the trigger threshold.

- FastAPI also exposes proxy routes for intervention stats, history, and health, so the prediction API acts as the primary integration point for the platform.

12. External dependencies that matter architecturally

- Mistral AI is used in both services, but for different concerns: explainability in the prediction service and message generation in the intervention service.

- Gmail SMTP is the actual outbound delivery channel, and Docker is used for PostgreSQL and the Python deployment path.

- The whole system is designed around batch feature computation plus low-latency online scoring and automated outbound intervention.