# shyamganeshd2006/manufacturing_defect_drift_detection architecture

> Architecture diagram of the GitHub repository shyamganeshd2006/manufacturing_defect_drift_detection, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-04). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/shyamganeshd2006/manufacturing_defect_drift_detection
- Repository: https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection

## Overview

This is a real-time industrial defect-prediction and drift-monitoring application. Sensor events are generated or produced to Kafka, processed by Spark per machine, scored by the model and checked for drift and risk, then written to PostgreSQL/Supabase for a FastAPI-backed React dashboard. The dashboard provides fleet overview, machine detail, drift monitoring, trends, and prediction history. Some streaming and API wiring is supported by the source index rather than excerpts; those connections are shown only where indexed references or the application’s documented pipeline support them.

Main entry points: [`spark_streaming/machine_state.py`](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/spark_streaming/machine_state.py) (Machine state analysis), [`frontend/src/App.jsx`](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/App.jsx) (Dashboard routes), [`frontend/src/api.js`](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/api.js) (API polling client), [`frontend/src/pages/MachineDetail.jsx`](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/MachineDetail.jsx) (Machine detail)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Event ingestion"]
  node_generator["Synthetic generator"]
  node_producer["Kafka producer"]
  node_kafka["Kafka"]
  node_spark["Streaming job<br/>[stream_job.py]"]
end

subgraph group_analysis["Prediction and monitoring"]
  node_machine["Machine state analysis<br/>[machine_state.py]"]
  node_features["Feature engineering<br/>[common.py]"]
  node_model["Defect model<br/>[train_model.py]"]
  node_explain["Root cause analysis<br/>[explain.py]"]
  node_behavior["Adaptive behavior<br/>[adaptive_window.py]"]
  node_drift["Drift detection<br/>[drift_detector.py]"]
  node_driftstats["Drift statistics<br/>[drift_stats.py]"]
  node_crossverify["Risk verification<br/>[cross_verify.py]"]
end

subgraph group_persistence["Data and API"]
  node_database[("Prediction store<br/>[supabase_client.py]")]
  node_postgres[("PostgreSQL")]
  node_api["FastAPI service<br/>[main.py]"]
  node_config["Shared configuration<br/>[config.py]"]
end

subgraph group_interface["Dashboard"]
  node_app["Dashboard routes<br/>[App.jsx]"]
  node_client["API polling client<br/>[api.js]"]
  node_overview["Fleet overview<br/>[Overview.jsx]"]
  node_detail["Machine detail<br/>[MachineDetail.jsx]"]
  node_driftpage["Drift monitor<br/>[DriftMonitor.jsx]"]
  node_trends["Trend analysis<br/>[Trends.jsx]"]
  node_history["Prediction history<br/>[History.jsx]"]
  node_charts["Charts<br/>[charts.jsx]"]
  node_common["Shared components<br/>[common.jsx]"]
end

node_operator(("Operator"))

node_operator -->|"views dashboard"| node_app
node_generator -->|"supplies events"| node_producer
node_producer -.->|"publishes events"| node_kafka
node_kafka -.->|"streams events"| node_spark
node_spark -->|"processes batches"| node_machine
node_spark -->|"writes predictions"| node_database
node_machine -->|"uses features"| node_features
node_machine -->|"uses adaptive window"| node_behavior
node_machine -->|"computes drift"| node_driftstats
node_machine -->|"verifies risk"| node_crossverify
node_machine -.->|"requests scoring"| node_model
node_machine -.->|"requests attribution"| node_explain
node_model -->|"builds features"| node_features
node_explain -->|"rebuilds features"| node_features
node_drift -->|"uses statistics"| node_driftstats
node_database -->|"reads and writes"| node_postgres
node_api -->|"queries data"| node_database
node_api -->|"reads settings"| node_config
node_database -->|"reads credentials"| node_config
node_app -->|"routes to"| node_overview
node_app -->|"routes to"| node_detail
node_app -->|"routes to"| node_driftpage
node_app -->|"routes to"| node_trends
node_app -->|"routes to"| node_history
node_overview -->|"polls data"| node_client
node_detail -->|"polls data"| node_client
node_driftpage -->|"polls data"| node_client
node_trends -->|"polls data"| node_client
node_history -->|"polls data"| node_client
node_client -.->|"requests endpoints"| node_api
node_detail -->|"renders charts"| node_charts
node_driftpage -->|"renders charts"| node_charts
node_trends -->|"renders charts"| node_charts
node_overview -->|"renders widgets"| node_common
node_detail -->|"renders widgets"| node_common
node_history -->|"renders widgets"| node_common

click node_generator "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/synthetic_generator.py"
click node_producer "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/synthetic_producer.py"
click node_spark "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/spark_streaming/stream_job.py"
click node_machine "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/spark_streaming/machine_state.py"
click node_features "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/model/common.py"
click node_model "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/model/train_model.py"
click node_explain "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/model/explain.py"
click node_behavior "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/spark_streaming/adaptive_window.py"
click node_drift "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/drift_detector.py"
click node_driftstats "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/drift_stats.py"
click node_crossverify "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/spark_streaming/cross_verify.py"
click node_database "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/database/supabase_client.py"
click node_api "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/api/main.py"
click node_config "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/config.py"
click node_app "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/App.jsx"
click node_client "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/api.js"
click node_overview "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/Overview.jsx"
click node_detail "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/MachineDetail.jsx"
click node_driftpage "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/DriftMonitor.jsx"
click node_trends "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/Trends.jsx"
click node_history "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/History.jsx"
click node_charts "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/components/charts.jsx"
click node_common "https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/components/common.jsx"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_generator,node_producer,node_kafka,node_spark toneBlue
class node_machine,node_features,node_model,node_explain,node_behavior,node_drift,node_driftstats,node_crossverify toneAmber
class node_database,node_postgres,node_api,node_config toneMint
class node_app,node_client,node_overview,node_detail,node_driftpage,node_trends,node_history,node_charts,node_common toneRose
class node_operator toneIndigo
```

## Components

### Event ingestion

- [Synthetic generator](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/synthetic_generator.py): `synthetic_generator.py`
- [Kafka producer](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/synthetic_producer.py): `synthetic_producer.py`
- **Kafka**
- [Streaming job](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/spark_streaming/stream_job.py): `spark_streaming/stream_job.py`

### Prediction and monitoring

- [Machine state analysis](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/spark_streaming/machine_state.py): `spark_streaming/machine_state.py`
- [Feature engineering](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/model/common.py): `model/common.py`
- [Defect model](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/model/train_model.py): `model/train_model.py`
- [Root cause analysis](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/model/explain.py): `model/explain.py`
- [Adaptive behavior](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/spark_streaming/adaptive_window.py): `spark_streaming/adaptive_window.py`
- [Drift detection](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/drift_detector.py): `drift_detector.py`
- [Drift statistics](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/drift_stats.py): `drift_stats.py`
- [Risk verification](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/spark_streaming/cross_verify.py): `spark_streaming/cross_verify.py`

### Data and API

- [Prediction store](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/database/supabase_client.py): `database/supabase_client.py`
- **PostgreSQL**
- [FastAPI service](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/api/main.py): `api/main.py`
- [Shared configuration](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/config.py): `config.py`

### Dashboard

- [Dashboard routes](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/App.jsx): `frontend/src/App.jsx`
- [API polling client](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/api.js): `frontend/src/api.js`
- [Fleet overview](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/Overview.jsx): `frontend/src/pages/Overview.jsx`
- [Machine detail](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/MachineDetail.jsx): `frontend/src/pages/MachineDetail.jsx`
- [Drift monitor](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/DriftMonitor.jsx): `frontend/src/pages/DriftMonitor.jsx`
- [Trend analysis](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/Trends.jsx): `frontend/src/pages/Trends.jsx`
- [Prediction history](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/pages/History.jsx): `frontend/src/pages/History.jsx`
- [Charts](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/components/charts.jsx): `frontend/src/components/charts.jsx`
- [Shared components](https://github.com/shyamganeshd2006/manufacturing_defect_drift_detection/blob/main/frontend/src/components/common.jsx): `frontend/src/components/common.jsx`

### Other components

- **Operator**

## Connections

- Operator → Dashboard routes: views dashboard (evidence: `frontend/src/App.jsx`)
- Synthetic generator → Kafka producer: supplies events (evidence: `synthetic_producer.py`)
- Kafka producer → Kafka: publishes events
- Kafka → Streaming job: streams events
- Streaming job → Machine state analysis: processes batches (evidence: `spark_streaming/stream_job.py`)
- Streaming job → Prediction store: writes predictions (evidence: `spark_streaming/stream_job.py`)
- Machine state analysis → Feature engineering: uses features (evidence: `spark_streaming/machine_state.py`)
- Machine state analysis → Adaptive behavior: uses adaptive window (evidence: `spark_streaming/machine_state.py`)
- Machine state analysis → Drift statistics: computes drift (evidence: `spark_streaming/machine_state.py`)
- Machine state analysis → Risk verification: verifies risk (evidence: `spark_streaming/machine_state.py`)
- Machine state analysis → Defect model: requests scoring
- Machine state analysis → Root cause analysis: requests attribution
- Defect model → Feature engineering: builds features (evidence: `model/train_model.py`)
- Root cause analysis → Feature engineering: rebuilds features (evidence: `model/explain.py`)
- Drift detection → Drift statistics: uses statistics (evidence: `drift_detector.py`)
- Prediction store → PostgreSQL: reads and writes (evidence: `database/supabase_client.py`)
- FastAPI service → Prediction store: queries data (evidence: `api/main.py`)
- FastAPI service → Shared configuration: reads settings (evidence: `api/main.py`)
- Prediction store → Shared configuration: reads credentials (evidence: `database/supabase_client.py`)
- Dashboard routes → Fleet overview: routes to (evidence: `frontend/src/App.jsx`)
- Dashboard routes → Machine detail: routes to (evidence: `frontend/src/App.jsx`)
- Dashboard routes → Drift monitor: routes to (evidence: `frontend/src/App.jsx`)
- Dashboard routes → Trend analysis: routes to (evidence: `frontend/src/App.jsx`)
- Dashboard routes → Prediction history: routes to (evidence: `frontend/src/App.jsx`)
- Fleet overview → API polling client: polls data (evidence: `frontend/src/pages/Overview.jsx`)
- Machine detail → API polling client: polls data (evidence: `frontend/src/pages/MachineDetail.jsx`)
- Drift monitor → API polling client: polls data (evidence: `frontend/src/pages/DriftMonitor.jsx`)
- Trend analysis → API polling client: polls data (evidence: `frontend/src/pages/Trends.jsx`)
- Prediction history → API polling client: polls data (evidence: `frontend/src/pages/History.jsx`)
- API polling client → FastAPI service: requests endpoints
- Machine detail → Charts: renders charts (evidence: `frontend/src/pages/MachineDetail.jsx`)
- Drift monitor → Charts: renders charts (evidence: `frontend/src/pages/DriftMonitor.jsx`)
- Trend analysis → Charts: renders charts (evidence: `frontend/src/pages/Trends.jsx`)
- Fleet overview → Shared components: renders widgets (evidence: `frontend/src/pages/Overview.jsx`)
- Machine detail → Shared components: renders widgets (evidence: `frontend/src/pages/MachineDetail.jsx`)
- Prediction history → Shared components: renders widgets (evidence: `frontend/src/pages/History.jsx`)
