How douglas-moore-engineering/velaqua-water-intelligence-platform works
Velaqua turns water-network telemetry into validated datasets, leak-risk predictions, prioritized incidents, and evidence-backed operational guidance subject to human approval. The documented workflow spans batch and streaming acquisition, Databricks bronze/silver/gold processing, Azure ML training and serving, FastAPI decisioning, and optional RAG and operational integrations. Source excerpts confirm several transformations and service behaviors; broader links follow the README’s architecture where code was not sampled.
Main entry points: app/main.py Decision API · bronze_ingest.py Bronze ingestion · app/main.py RAG service · deployment AML endpoints
Components 21
Telemetry data
- ADF ingestion
platform/ingestion/adf - Event Hub consumer
platform/streaming/eventhub_consumer.py - ADLS raw data
- Bronze ingestion
platform/databricks/jobs/bronze_ingest.py - Quality validation
platform/databricks/jobs/data_quality_validate.py - Silver transforms
platform/databricks/jobs/silver_transform.py - Gold dataset
platform/databricks/jobs/gold_build.py
ML lifecycle
- Model training
ml/training/src/train_model_suite.py - Candidate selection
ml/training/src/select_production_candidate.py - AML endpoints
ml/deployment - Inference scoring
ml/deployment/score.py
Decision services
- Decision API
services/decision_api/app/main.py - Decision engine
services/decision_api/app/decision_engine.py - Shared contracts
services/shared/models.py
Evidence and operations
- RAG service
services/rag_service/app/main.py - Azure AI Search
- Azure OpenAI
- Field and CMMS workflows
services/decision_api/app/main.py - Monitoring and governance
governance
Other components
- Sensors / SCADA
- Human operator
Connections 22
- Sensors / SCADA → ADF ingestion: batch ingest
- Sensors / SCADA → Event Hub consumer: stream events
- ADF ingestion → ADLS raw data: write raw
- Event Hub consumer → ADLS raw data: write events
- ADLS raw data → Bronze ingestion: read telemetry
- Bronze ingestion → Quality validation: validate data
- Bronze ingestion → Silver transforms: transform data
- Silver transforms → Gold dataset: build dataset
- Gold dataset → Model training: train from
- Model training → Candidate selection: evaluate candidates
- Candidate selection → AML endpoints: register candidate
- AML endpoints → Inference scoring: serve model
- Decision API → AML endpoints: request predictions
- Decision API → Decision engine: compute decision
- Decision engine → Shared contracts: return decision
- Decision API → RAG service: request evidence
- RAG service → Azure AI Search: retrieve documents
- RAG service → Azure OpenAI: synthesize evidence
- Decision API → Human operator: request approval
- Human operator → Decision API: record approval
- Decision API → Field and CMMS workflows: dispatch actions
- Decision API → Monitoring and governance: track outcomes