How shyamganeshd2006/manufacturing_defect_drift_detection works
This is a real-time industrial defect-prediction and drift-monitoring application. Sensor events are generated or produced to Kafka, processed by Spark per machine, scored by the model and checked for drift and risk, then written to PostgreSQL/Supabase for a FastAPI-backed React dashboard. The dashboard provides fleet overview, machine detail, drift monitoring, trends, and prediction history. Some streaming and API wiring is supported by the source index rather than excerpts; those connections are shown only where indexed references or the application’s documented pipeline support them.
Main entry points: machine_state.py Machine state analysis · App.jsx Dashboard routes · api.js API polling client · MachineDetail.jsx Machine detail
Components 26
Event ingestion
- Synthetic generator
synthetic_generator.py - Kafka producer
synthetic_producer.py - Kafka
- Streaming job
spark_streaming/stream_job.py
Prediction and monitoring
- Machine state analysis
spark_streaming/machine_state.py - Feature engineering
model/common.py - Defect model
model/train_model.py - Root cause analysis
model/explain.py - Adaptive behavior
spark_streaming/adaptive_window.py - Drift detection
drift_detector.py - Drift statistics
drift_stats.py - Risk verification
spark_streaming/cross_verify.py
Data and API
- Prediction store
database/supabase_client.py - PostgreSQL
- FastAPI service
api/main.py - Shared configuration
config.py
Dashboard
- Dashboard routes
frontend/src/App.jsx - API polling client
frontend/src/api.js - Fleet overview
frontend/src/pages/Overview.jsx - Machine detail
frontend/src/pages/MachineDetail.jsx - Drift monitor
frontend/src/pages/DriftMonitor.jsx - Trend analysis
frontend/src/pages/Trends.jsx - Prediction history
frontend/src/pages/History.jsx - Charts
frontend/src/components/charts.jsx - Shared components
frontend/src/components/common.jsx
Other components
- Operator
Connections 36
- Operator → Dashboard routes: views dashboard
frontend/src/App.jsx - Synthetic generator → Kafka producer: supplies events
synthetic_producer.py - Kafka producer → Kafka: publishes events
- Kafka → Streaming job: streams events
- Streaming job → Machine state analysis: processes batches
spark_streaming/stream_job.py - Streaming job → Prediction store: writes predictions
spark_streaming/stream_job.py - Machine state analysis → Feature engineering: uses features
spark_streaming/machine_state.py - Machine state analysis → Adaptive behavior: uses adaptive window
spark_streaming/machine_state.py - Machine state analysis → Drift statistics: computes drift
spark_streaming/machine_state.py - Machine state analysis → Risk verification: verifies risk
spark_streaming/machine_state.py - Machine state analysis → Defect model: requests scoring
- Machine state analysis → Root cause analysis: requests attribution
- Defect model → Feature engineering: builds features
model/train_model.py - Root cause analysis → Feature engineering: rebuilds features
model/explain.py - Drift detection → Drift statistics: uses statistics
drift_detector.py - Prediction store → PostgreSQL: reads and writes
database/supabase_client.py - FastAPI service → Prediction store: queries data
api/main.py - FastAPI service → Shared configuration: reads settings
api/main.py - Prediction store → Shared configuration: reads credentials
database/supabase_client.py - Dashboard routes → Fleet overview: routes to
frontend/src/App.jsx - Dashboard routes → Machine detail: routes to
frontend/src/App.jsx - Dashboard routes → Drift monitor: routes to
frontend/src/App.jsx - Dashboard routes → Trend analysis: routes to
frontend/src/App.jsx - Dashboard routes → Prediction history: routes to
frontend/src/App.jsx - Fleet overview → API polling client: polls data
frontend/src/pages/Overview.jsx - Machine detail → API polling client: polls data
frontend/src/pages/MachineDetail.jsx - Drift monitor → API polling client: polls data
frontend/src/pages/DriftMonitor.jsx - Trend analysis → API polling client: polls data
frontend/src/pages/Trends.jsx - Prediction history → API polling client: polls data
frontend/src/pages/History.jsx - API polling client → FastAPI service: requests endpoints
- Machine detail → Charts: renders charts
frontend/src/pages/MachineDetail.jsx - Drift monitor → Charts: renders charts
frontend/src/pages/DriftMonitor.jsx - Trend analysis → Charts: renders charts
frontend/src/pages/Trends.jsx - Fleet overview → Shared components: renders widgets
frontend/src/pages/Overview.jsx - Machine detail → Shared components: renders widgets
frontend/src/pages/MachineDetail.jsx - Prediction history → Shared components: renders widgets
frontend/src/pages/History.jsx