How ngnquanq/credit-risk-ml-system works
This is a credit-risk decisioning platform: a loan is submitted through the API, persisted, enriched through CDC-driven feature streams, materialized in Feast/Redis, and scored by a model service; the score is published to Kafka. Training and registration form a separate model-lifecycle branch. The README documents the end-to-end stream topology, while sampled source confirms API/workflow structure, persistence, Feast setup, and scoring behavior; links within unsampled stream and deployment components follow the documented architecture.
Main entry points: repository.py Feast Feature Store · service.py BentoML Scoring · feature_consumer.py DWH Feature Consumer · main.py Loan API
Components 22
Loan Intake
- Loan API
application/entrypoints/api/main.py - Submit Workflow
application/workflows/submit_loan.py - PostgreSQL
Feature Engineering
- Debezium CDC
- Kafka Topics
- Bureau Consumer
application/entrypoints/bureau_consumer.py - DWH Feature Consumer
application/entrypoints/feature_consumer.py - Bureau Aggregation
application/flink/jobs/bureau_aggregation_etl.py - Application CDC ETL
application/flink/jobs/cdc_application_etl.py - Feast Feature Store
application/feast_repo/repository.py - Redis Online Store
- ClickHouse Sources
Online Decisioning
- Feature Ready Event
- Knative Sequence
- BentoML Scoring
application/scoring/service.py - Scoring Topic
Model Lifecycle
- Registered Model
application/scoring/model_registry.py - Training Jobs
application/training/train_register.py - Kubeflow Pipeline
platform/ml/k8s/training-pipeline/pipeline.py - MLflow Registry
User Experience
- Frontend Dashboard
application/frontend/frontend.py
Other components
- Applicant
Connections 25
- Applicant → Loan API: submits application
- Loan API → Submit Workflow: delegates submission
- Submit Workflow → PostgreSQL: persists application
- PostgreSQL → Debezium CDC: captures changes
- Debezium CDC → Kafka Topics: publishes changes
- Kafka Topics → Application CDC ETL: streams application data
- Application CDC ETL → Feast Feature Store: produces features
- Kafka Topics → Bureau Consumer: streams bureau events
- Bureau Consumer → Bureau Aggregation: forwards bureau data
- Bureau Aggregation → Feast Feature Store: produces aggregates
- Kafka Topics → DWH Feature Consumer: streams feature events
- DWH Feature Consumer → ClickHouse Sources: fetches DWH data
- ClickHouse Sources → DWH Feature Consumer: returns records
- DWH Feature Consumer → Feast Feature Store: produces DWH features
- Feast Feature Store → Redis Online Store: materializes features
- Feast Feature Store → Feature Ready Event: emits readiness event
- Feature Ready Event → Knative Sequence: triggers scoring
- Knative Sequence → BentoML Scoring: dispatches event
- BentoML Scoring → Feast Feature Store: reads online features
- BentoML Scoring → Registered Model: loads model
- BentoML Scoring → Scoring Topic: publishes prediction
- Training Jobs → MLflow Registry: registers model
- Kubeflow Pipeline → MLflow Registry: trains and registers
- MLflow Registry → Registered Model: stores artifacts
- Frontend Dashboard → Loan API: uses API