# akilanpl/survey-data-intelligence architecture

> Architecture diagram of the GitHub repository akilanpl/survey-data-intelligence, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/akilanpl/survey-data-intelligence
- Repository: https://github.com/akilanpl/survey-data-intelligence

## Overview

SIRL turns survey submissions into normalized, profiled batches, then runs deterministic rules, statistics, and ML validation before fusing evidence into risk assessments and classifications. Optional AI explains selected assessments but cannot affect scoring. Supervisors review results in the dashboard and manage investigations. The README documents the end-to-end stages; sampled code confirms selected engine, dashboard, explanation, and investigation behaviors, so uncertain internal wiring is omitted.

Main entry points: [`backend/app/modules/pipeline/orchestrator.py`](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/pipeline/orchestrator.py) (Pipeline orchestration), [`backend/app/main.py`](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/main.py) (API application), [`backend/app/db.py`](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/db.py) (SQLite state), [`backend/app/modules/storage/parquet.py`](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/storage/parquet.py) (Batch Parquet)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ui["Supervisor interface"]
  node_web["Dashboard UI"]
  node_auth["Authentication"]
  node_dashboard["Dashboard queries<br/>[service.py]"]
end

subgraph group_ingest["Ingestion and pipeline"]
  node_ingestion["Input standardization"]
  node_pipeline["Pipeline orchestration<br/>[orchestrator.py]"]
  node_parquet[("Batch Parquet<br/>[parquet.py]")]
end

subgraph group_analysis["Profiling and validation"]
  node_sirl["SIRL profiling"]
  node_rules["Rule validation<br/>[engine.py]"]
  node_statistics["Statistical validation<br/>[engine.py]"]
  node_ml["ML validation<br/>[engine.py]"]
  node_fusion["Evidence fusion<br/>[engine.py]"]
  node_intelligence["Intelligence analysis"]
end

subgraph group_review["Explanation and review"]
  node_explanation["Assessment explanations<br/>[service.py]"]
  node_ai["AI provider layer"]
  node_investigation["Case investigations<br/>[service.py]"]
end

subgraph group_state["Platform state"]
  node_api["API application<br/>[main.py]"]
  node_database[("SQLite state<br/>[db.py]")]
end

node_supervisor(("Supervisor"))
node_csv["CSV submission"]
node_esigma(("eSIGMA source"))
node_ai_vendor(("AI provider"))

node_supervisor -->|"uses"| node_web
node_csv -->|"submits"| node_ingestion
node_esigma -.->|"provides data"| node_ingestion
node_web -->|"requests"| node_api
node_api -->|"dispatches"| node_auth
node_api -->|"dispatches"| node_ingestion
node_api -->|"dispatches"| node_pipeline
node_api -->|"dispatches"| node_dashboard
node_api -->|"dispatches"| node_investigation
node_ingestion -->|"writes batch"| node_parquet
node_pipeline -->|"runs profiling"| node_sirl
node_pipeline -->|"runs validation"| node_rules
node_pipeline -->|"runs validation"| node_statistics
node_pipeline -->|"runs validation"| node_ml
node_pipeline -->|"runs fusion"| node_fusion
node_sirl -->|"reads batch"| node_parquet
node_statistics -->|"reads batch"| node_parquet
node_ml -->|"reads batch"| node_parquet
node_fusion -->|"reads records"| node_parquet
node_fusion -->|"classifies"| node_intelligence
node_rules -->|"persists evidence"| node_database
node_statistics -->|"persists evidence"| node_database
node_ml -->|"persists evidence"| node_database
node_fusion -->|"persists assessments"| node_database
node_explanation -->|"reads assessments"| node_database
node_explanation -->|"requests explanation"| node_ai
node_ai -.->|"calls provider"| node_ai_vendor
node_dashboard -->|"reads results"| node_database
node_investigation -->|"stores cases and audit"| node_database
node_web -->|"manages cases"| node_investigation

click node_web "https://github.com/akilanpl/survey-data-intelligence/tree/main/frontend/app/dashboard"
click node_api "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/main.py"
click node_auth "https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/auth"
click node_dashboard "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/dashboard/service.py"
click node_ingestion "https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/ingestion"
click node_pipeline "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/pipeline/orchestrator.py"
click node_parquet "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/storage/parquet.py"
click node_sirl "https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/sirl"
click node_rules "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/rules/engine.py"
click node_statistics "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/statistics/engine.py"
click node_ml "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/ml/engine.py"
click node_fusion "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/fusion/engine.py"
click node_intelligence "https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/validation/intelligence"
click node_explanation "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/explanation/service.py"
click node_ai "https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/ai"
click node_investigation "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/investigations/service.py"
click node_database "https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/db.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web,node_auth,node_dashboard toneBlue
class node_ingestion,node_pipeline,node_parquet toneAmber
class node_sirl,node_rules,node_statistics,node_ml,node_fusion,node_intelligence toneMint
class node_explanation,node_ai,node_investigation toneRose
class node_api,node_database,node_supervisor,node_csv,node_esigma,node_ai_vendor toneIndigo
```

## Components

### Supervisor interface

- [Dashboard UI](https://github.com/akilanpl/survey-data-intelligence/tree/main/frontend/app/dashboard): `frontend/app/dashboard`
- [Authentication](https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/auth): `backend/app/modules/auth`
- [Dashboard queries](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/dashboard/service.py): `backend/app/modules/dashboard/service.py`

### Ingestion and pipeline

- [Input standardization](https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/ingestion): `backend/app/modules/ingestion`
- [Pipeline orchestration](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/pipeline/orchestrator.py): `backend/app/modules/pipeline/orchestrator.py`
- [Batch Parquet](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/storage/parquet.py): `backend/app/modules/storage/parquet.py`

### Profiling and validation

- [SIRL profiling](https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/sirl): `backend/app/modules/sirl`
- [Rule validation](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/rules/engine.py): `backend/app/modules/validation/rules/engine.py`
- [Statistical validation](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/statistics/engine.py): `backend/app/modules/validation/statistics/engine.py`
- [ML validation](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/ml/engine.py): `backend/app/modules/validation/ml/engine.py`
- [Evidence fusion](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/fusion/engine.py): `backend/app/modules/validation/fusion/engine.py`
- [Intelligence analysis](https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/validation/intelligence): `backend/app/modules/validation/intelligence`

### Explanation and review

- [Assessment explanations](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/validation/explanation/service.py): `backend/app/modules/validation/explanation/service.py`
- [AI provider layer](https://github.com/akilanpl/survey-data-intelligence/tree/main/backend/app/modules/ai): `backend/app/modules/ai`
- [Case investigations](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/modules/investigations/service.py): `backend/app/modules/investigations/service.py`

### Platform state

- [API application](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/main.py): `backend/app/main.py`
- [SQLite state](https://github.com/akilanpl/survey-data-intelligence/blob/main/backend/app/db.py): `backend/app/db.py`

### Other components

- **Supervisor**
- **CSV submission**
- **eSIGMA source**
- **AI provider**

## Connections

- Supervisor → Dashboard UI: uses
- CSV submission → Input standardization: submits
- eSIGMA source → Input standardization: provides data
- Dashboard UI → API application: requests
- API application → Authentication: dispatches
- API application → Input standardization: dispatches
- API application → Pipeline orchestration: dispatches
- API application → Dashboard queries: dispatches
- API application → Case investigations: dispatches
- Input standardization → Batch Parquet: writes batch
- Pipeline orchestration → SIRL profiling: runs profiling
- Pipeline orchestration → Rule validation: runs validation
- Pipeline orchestration → Statistical validation: runs validation
- Pipeline orchestration → ML validation: runs validation
- Pipeline orchestration → Evidence fusion: runs fusion
- SIRL profiling → Batch Parquet: reads batch
- Statistical validation → Batch Parquet: reads batch
- ML validation → Batch Parquet: reads batch
- Evidence fusion → Batch Parquet: reads records
- Evidence fusion → Intelligence analysis: classifies
- Rule validation → SQLite state: persists evidence
- Statistical validation → SQLite state: persists evidence
- ML validation → SQLite state: persists evidence
- Evidence fusion → SQLite state: persists assessments
- Assessment explanations → SQLite state: reads assessments
- Assessment explanations → AI provider layer: requests explanation
- AI provider layer → AI provider: calls provider
- Dashboard queries → SQLite state: reads results
- Case investigations → SQLite state: stores cases and audit
- Dashboard UI → Case investigations: manages cases