How shazfa13/fraud-detection-system works
This repository is a PaySim fraud-detection project spanning exploratory analysis, past-only feature engineering, chronological Model V1 training, an HTTP prediction API, and a Kafka/Spark streaming path. The documented Phase 6 simulation warms account history before sending live requests; the streaming producer replays transactions without labels, and Spark reuses the API’s feature and model logic. The README describes streaming as not implemented, but the supplied streaming source files implement it. Notebook-stage links reflect README documentation; unsampled notebook internals are not inferred.
Main entry points: app.py Prediction API · spark_streaming.py Spark stream consumer · 04_xgboost_model_v1.ipynb Model V1 training · kafka_producer.py Transaction producer
Components 21
Offline modeling
- Exploratory analysis
notebooks/02_exploratory_data_analysis.ipynb - Feature engineering
notebooks/03_feature_engineering.ipynb - Model V1 training
notebooks/04_xgboost_model_v1.ipynb - API deployment
notebooks/05_model_deployment.ipynb
Prediction serving
- Prediction API
src/api/app.py - Account history
src/api/app.py - Feature pipeline
src/api/app.py - Live simulation
notebooks/06_live_transaction_simulation.ipynb
Streaming pipeline
- Transaction producer
src/streaming/kafka_producer.py - Streaming configuration
src/streaming/config.py - Local bridge inbox
- Spark stream consumer
src/streaming/spark_streaming.py
Data and outputs
- PaySim dataset
- XGBoost Model V1
models/xgboost_model_v1.json - Model metadata
models/model_v1_metadata.json - Risk threshold
models/model_v1_threshold.json - Streaming predictions
src/streaming/config.py - Live evaluation outputs
reports
Other components
- Project analyst
- API client
- Kafka broker
Connections 23
- Project analyst → Exploratory analysis: explores data
README.md - PaySim dataset → Exploratory analysis: analyzes
- Exploratory analysis → Feature engineering: informs
- Feature engineering → Model V1 training: produces features
- Model V1 training → XGBoost Model V1: trains
- Model V1 training → Model metadata: records feature names
- Model V1 training → Risk threshold: selects threshold
- API deployment → Prediction API: deploys
README.md - API client → Prediction API: requests prediction
src/api/app.py - Prediction API → Account history: reads and updates
src/api/app.py - Prediction API → Feature pipeline: builds features
src/api/app.py - Feature pipeline → XGBoost Model V1: scores with
src/api/app.py - Live simulation → Prediction API: sends live requests
README.md - Live simulation → Live evaluation outputs: writes evaluation data
README.md - Streaming configuration → Transaction producer: configures
src/streaming/kafka_producer.py - Streaming configuration → Spark stream consumer: configures
src/streaming/spark_streaming.py - Transaction producer → PaySim dataset: replays transactions
src/streaming/kafka_producer.py - Transaction producer → Kafka broker: publishes events
src/streaming/kafka_producer.py - Transaction producer → Local bridge inbox: writes events
src/streaming/kafka_producer.py - Kafka broker → Spark stream consumer: streams events
src/streaming/spark_streaming.py - Local bridge inbox → Spark stream consumer: streams files
src/streaming/spark_streaming.py - Spark stream consumer → Prediction API: reuses feature and model logic
src/streaming/spark_streaming.py - Spark stream consumer → Streaming predictions: appends predictions
src/streaming/spark_streaming.py