# bhagyesh-074/fraudstream architecture

> Architecture diagram of the GitHub repository bhagyesh-074/fraudstream, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhagyesh-074/fraudstream
- Repository: https://github.com/bhagyesh-074/fraudstream

## Overview

FraudStream detects credit-card fraud through offline model development and Kafka-based real-time scoring. The primary runtime path replays transactions into Kafka, scores each event with shared feature logic and an XGBoost model, publishes scored events, and exposes telemetry. Separate branches ingest raw events into a Parquet lake and monitor feature drift; Airflow can trigger retraining and registry promotion gating. The README documents several integrations and flows; source excerpts are partial, so uncertain internal links are omitted.

Main entry points: [`src/pipeline/train.py`](https://github.com/bhagyesh-074/fraudstream/blob/main/src/pipeline/train.py) (Model training), [`src/eda/run_eda.py`](https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/run_eda.py) (EDA runner), [`src/serving/scoring_consumer.py`](https://github.com/bhagyesh-074/fraudstream/blob/main/src/serving/scoring_consumer.py) (Scoring consumer), [`src/orchestration/retrain_dag.py`](https://github.com/bhagyesh-074/fraudstream/blob/main/src/orchestration/retrain_dag.py) (Retraining DAG)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_eda["Data exploration"]
  node_download["Dataset check<br/>[download_data.py]"]
  node_profiling["Data profiling<br/>[profiling.py]"]
  node_imbalance["Class imbalance<br/>[imbalance.py]"]
  node_costmatrix["Fraud cost model<br/>[cost_matrix.py]"]
  node_eda_runner["EDA runner<br/>[run_eda.py]"]
end

subgraph group_training["Model development"]
  node_cleaning["Data cleaning<br/>[cleaning.py]"]
  node_feature_logic["Shared features<br/>[feature_logic.py]"]
  node_feature_pipeline["Training features<br/>[features.py]"]
  node_feature_store[("Feature store<br/>[feature_store.py]")]
  node_evaluation["Model evaluation<br/>[evaluate.py]"]
  node_training_pipeline["Model training<br/>[train.py]"]
  node_model_file["Champion model"]
end

subgraph group_streaming["Event processing"]
  node_producer["Kafka producer<br/>[producer.py]"]
  node_kafka["Kafka broker"]
  node_raw_consumer["Raw event consumer<br/>[raw_consumer.py]"]
  node_raw_lake[("Partitioned Parquet lake")]
end

subgraph group_serving["Scoring and telemetry"]
  node_loader["Model loader<br/>[model_loader.py]"]
  node_scorer["Scoring consumer"]
  node_scored_topic["Scored events"]
  node_api["Telemetry API<br/>[api.py]"]
end

subgraph group_operations["Monitoring and retraining"]
  node_drift["Drift detector<br/>[drift_detector.py]"]
  node_airflow["Retraining DAG<br/>[retrain_dag.py]"]
  node_registry[("MLflow registry")]
end

node_dataset[("Transaction dataset")]
node_operator(("Operator"))

node_dataset -->|"checks availability"| node_download
node_eda_runner -->|"checks dataset"| node_download
node_eda_runner -->|"profiles data"| node_profiling
node_eda_runner -->|"reports imbalance"| node_imbalance
node_eda_runner -->|"calibrates costs"| node_costmatrix
node_operator -->|"runs analysis"| node_eda_runner
node_operator -->|"trains model"| node_training_pipeline
node_training_pipeline -->|"uses cleaning"| node_cleaning
node_training_pipeline -->|"uses features"| node_feature_pipeline
node_training_pipeline -->|"evaluates models"| node_evaluation
node_feature_pipeline -->|"prepares features"| node_cleaning
node_cleaning -->|"computes features"| node_feature_logic
node_training_pipeline -->|"uses profiling"| node_profiling
node_training_pipeline -->|"uses cost model"| node_costmatrix
node_training_pipeline -->|"writes feature data"| node_feature_store
node_training_pipeline -->|"saves champion"| node_model_file
node_operator -->|"replays transactions"| node_producer
node_producer -->|"publishes raw events"| node_kafka
node_kafka -->|"delivers raw events"| node_raw_consumer
node_raw_consumer -->|"writes partitions"| node_raw_lake
node_kafka -->|"delivers transactions"| node_scorer
node_scorer -->|"computes features"| node_feature_logic
node_scorer -.->|"loads model"| node_loader
node_scorer -->|"publishes scores"| node_scored_topic
node_api -->|"serves scoring stats"| node_scorer
node_api -->|"reports model status"| node_loader
node_operator -->|"queries telemetry"| node_api
node_feature_store -->|"provides baseline"| node_drift
node_operator -->|"runs drift check"| node_drift
node_drift -->|"triggers retraining"| node_airflow
node_airflow -->|"runs drift check"| node_drift
node_airflow -->|"trains candidate"| node_training_pipeline
node_airflow -->|"gates promotion"| node_registry
node_training_pipeline -->|"registers candidate"| node_registry
node_registry -.->|"supplies production model"| node_loader

click node_download "https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/download_data.py"
click node_profiling "https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/profiling.py"
click node_imbalance "https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/imbalance.py"
click node_costmatrix "https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/cost_matrix.py"
click node_eda_runner "https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/run_eda.py"
click node_cleaning "https://github.com/bhagyesh-074/fraudstream/blob/main/src/pipeline/cleaning.py"
click node_feature_logic "https://github.com/bhagyesh-074/fraudstream/blob/main/src/features/feature_logic.py"
click node_feature_pipeline "https://github.com/bhagyesh-074/fraudstream/blob/main/src/pipeline/features.py"
click node_feature_store "https://github.com/bhagyesh-074/fraudstream/blob/main/src/features/feature_store.py"
click node_evaluation "https://github.com/bhagyesh-074/fraudstream/blob/main/src/pipeline/evaluate.py"
click node_training_pipeline "https://github.com/bhagyesh-074/fraudstream/blob/main/src/pipeline/train.py"
click node_model_file "https://github.com/bhagyesh-074/fraudstream/blob/main/models/xgboost_baseline.json.dvc"
click node_producer "https://github.com/bhagyesh-074/fraudstream/blob/main/src/streaming/producer.py"
click node_raw_consumer "https://github.com/bhagyesh-074/fraudstream/blob/main/src/streaming/raw_consumer.py"
click node_raw_lake "https://github.com/bhagyesh-074/fraudstream/tree/main/data"
click node_loader "https://github.com/bhagyesh-074/fraudstream/blob/main/src/serving/model_loader.py"
click node_scorer "https://github.com/bhagyesh-074/fraudstream/blob/main/src/serving/scoring_consumer.py"
click node_api "https://github.com/bhagyesh-074/fraudstream/blob/main/src/serving/api.py"
click node_drift "https://github.com/bhagyesh-074/fraudstream/blob/main/src/monitoring/drift_detector.py"
click node_airflow "https://github.com/bhagyesh-074/fraudstream/blob/main/src/orchestration/retrain_dag.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_download,node_profiling,node_imbalance,node_costmatrix,node_eda_runner toneBlue
class node_cleaning,node_feature_logic,node_feature_pipeline,node_feature_store,node_evaluation,node_training_pipeline,node_model_file,node_dataset toneAmber
class node_producer,node_kafka,node_raw_consumer,node_raw_lake toneMint
class node_loader,node_scorer,node_scored_topic,node_api toneRose
class node_drift,node_airflow,node_registry,node_operator toneIndigo
```

## Components

### Data exploration

- [Dataset check](https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/download_data.py): `src/eda/download_data.py`
- [Data profiling](https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/profiling.py): `src/eda/profiling.py`
- [Class imbalance](https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/imbalance.py): `src/eda/imbalance.py`
- [Fraud cost model](https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/cost_matrix.py): `src/eda/cost_matrix.py`
- [EDA runner](https://github.com/bhagyesh-074/fraudstream/blob/main/src/eda/run_eda.py): `src/eda/run_eda.py`

### Model development

- [Data cleaning](https://github.com/bhagyesh-074/fraudstream/blob/main/src/pipeline/cleaning.py): `src/pipeline/cleaning.py`
- [Shared features](https://github.com/bhagyesh-074/fraudstream/blob/main/src/features/feature_logic.py): `src/features/feature_logic.py`
- [Training features](https://github.com/bhagyesh-074/fraudstream/blob/main/src/pipeline/features.py): `src/pipeline/features.py`
- [Feature store](https://github.com/bhagyesh-074/fraudstream/blob/main/src/features/feature_store.py): `src/features/feature_store.py`
- [Model evaluation](https://github.com/bhagyesh-074/fraudstream/blob/main/src/pipeline/evaluate.py): `src/pipeline/evaluate.py`
- [Model training](https://github.com/bhagyesh-074/fraudstream/blob/main/src/pipeline/train.py): `src/pipeline/train.py`
- [Champion model](https://github.com/bhagyesh-074/fraudstream/blob/main/models/xgboost_baseline.json.dvc): `models/xgboost_baseline.json.dvc`

### Event processing

- [Kafka producer](https://github.com/bhagyesh-074/fraudstream/blob/main/src/streaming/producer.py): `src/streaming/producer.py`
- **Kafka broker**
- [Raw event consumer](https://github.com/bhagyesh-074/fraudstream/blob/main/src/streaming/raw_consumer.py): `src/streaming/raw_consumer.py`
- [Partitioned Parquet lake](https://github.com/bhagyesh-074/fraudstream/tree/main/data): `data`

### Scoring and telemetry

- [Model loader](https://github.com/bhagyesh-074/fraudstream/blob/main/src/serving/model_loader.py): `src/serving/model_loader.py`
- [Scoring consumer](https://github.com/bhagyesh-074/fraudstream/blob/main/src/serving/scoring_consumer.py): `src/serving/scoring_consumer.py`
- **Scored events**
- [Telemetry API](https://github.com/bhagyesh-074/fraudstream/blob/main/src/serving/api.py): `src/serving/api.py`

### Monitoring and retraining

- [Drift detector](https://github.com/bhagyesh-074/fraudstream/blob/main/src/monitoring/drift_detector.py): `src/monitoring/drift_detector.py`
- [Retraining DAG](https://github.com/bhagyesh-074/fraudstream/blob/main/src/orchestration/retrain_dag.py): `src/orchestration/retrain_dag.py`
- **MLflow registry**

### Other components

- **Transaction dataset**
- **Operator**

## Connections

- Transaction dataset → Dataset check: checks availability
- EDA runner → Dataset check: checks dataset (evidence: `src/eda/run_eda.py`)
- EDA runner → Data profiling: profiles data (evidence: `src/eda/run_eda.py`)
- EDA runner → Class imbalance: reports imbalance (evidence: `src/eda/run_eda.py`)
- EDA runner → Fraud cost model: calibrates costs (evidence: `src/eda/run_eda.py`)
- Operator → EDA runner: runs analysis (evidence: `README.md`)
- Operator → Model training: trains model (evidence: `README.md`)
- Model training → Data cleaning: uses cleaning (evidence: `src/pipeline/train.py`)
- Model training → Training features: uses features (evidence: `src/pipeline/train.py`)
- Model training → Model evaluation: evaluates models (evidence: `src/pipeline/train.py`)
- Training features → Data cleaning: prepares features (evidence: `src/pipeline/features.py`)
- Data cleaning → Shared features: computes features (evidence: `src/pipeline/cleaning.py`)
- Model training → Data profiling: uses profiling (evidence: `src/pipeline/train.py`)
- Model training → Fraud cost model: uses cost model (evidence: `src/pipeline/train.py`)
- Model training → Feature store: writes feature data (evidence: `README.md`)
- Model training → Champion model: saves champion (evidence: `README.md`)
- Operator → Kafka producer: replays transactions (evidence: `README.md`)
- Kafka producer → Kafka broker: publishes raw events (evidence: `README.md`)
- Kafka broker → Raw event consumer: delivers raw events (evidence: `README.md`)
- Raw event consumer → Partitioned Parquet lake: writes partitions (evidence: `README.md`)
- Kafka broker → Scoring consumer: delivers transactions (evidence: `README.md`)
- Scoring consumer → Shared features: computes features (evidence: `src/serving/scoring_consumer.py`)
- Scoring consumer → Model loader: loads model
- Scoring consumer → Scored events: publishes scores (evidence: `README.md`)
- Telemetry API → Scoring consumer: serves scoring stats (evidence: `src/serving/api.py`)
- Telemetry API → Model loader: reports model status (evidence: `src/serving/api.py`)
- Operator → Telemetry API: queries telemetry (evidence: `README.md`)
- Feature store → Drift detector: provides baseline (evidence: `README.md`)
- Operator → Drift detector: runs drift check (evidence: `README.md`)
- Drift detector → Retraining DAG: triggers retraining (evidence: `README.md`)
- Retraining DAG → Drift detector: runs drift check (evidence: `src/orchestration/retrain_dag.py`)
- Retraining DAG → Model training: trains candidate (evidence: `src/orchestration/retrain_dag.py`)
- Retraining DAG → MLflow registry: gates promotion (evidence: `README.md`)
- Model training → MLflow registry: registers candidate (evidence: `README.md`)
- MLflow registry → Model loader: supplies production model
