How zura16/clinical-flow works
ClinicalFlow is a synthetic healthcare-data lakehouse that moves FHIR, SQL Server EHR, and claims data through bronze ingestion, silver normalization and quality checks, and gold warehouse modeling for analytics. The main user workflow is to run the pipeline stages, then query warehouse outputs and operational dashboards. The graph also shows control metadata, quality/quarantine, audit, and alerting. Edges are limited to relationships documented in the README or visible in the SOURCE INDEX; unsampled capabilities are represented without inferred implementation links.
Main entry points: ingest_raw_data.py Bronze Ingestion · silver_engine.py Silver Engine · build_facts.py Fact Builder · failure_simulation.py Failure Recovery Demo
Components 26
Source Ingestion
- Bronze Ingestion
databricks/bronze/ingest_raw_data.py - Bronze Delta
databricks/utilities/config.py - SQL Server Access
databricks/utilities/sqlserver.py
Silver Processing
- FHIR Normalization
databricks/silver/process_fhir_silver.py - Relational Normalization
databricks/silver/process_relational_silver.py - Silver Engine
databricks/silver/silver_engine.py - Silver Mapping Specs
databricks/silver/specs.py - Bronze Reader
databricks/silver/bronze_reader.py - Quality and Quarantine
databricks/utilities/quality_engine.py - Silver Delta
databricks/utilities/config.py
Gold Warehouse
- Dimension Builder
databricks/gold/build_dimensions.py - Fact Builder
databricks/gold/build_facts.py - Silver Warehouse Reader
databricks/gold/silver_reader.py - Warehouse Keys
databricks/gold/keys.py - Gold Delta Warehouse
databricks/utilities/config.py
Control and Operations
- Ingestion Control
databricks/utilities/control.py - Run Audit
databricks/utilities/logger.py - Failure Alerts
databricks/utilities/alerting.py - Failure Recovery Demo
databricks/utilities/failure_simulation.py
Analytics
- Analytics Dashboards
dashboards
Other components
- Pipeline Operator
- FHIR Bundles
- SQL Server EHR
- Claims Feed
- Alert Webhook
- Analytics Consumer
Connections 33
- Pipeline Operator → Bronze Ingestion: runs ingestion
README.md - FHIR Bundles → Bronze Ingestion: supplies bundles
README.md - SQL Server EHR → Bronze Ingestion: supplies records
README.md - Claims Feed → Bronze Ingestion: supplies feed
README.md - Bronze Ingestion → Bronze Delta: writes raw data
README.md - Bronze Ingestion → Ingestion Control: uses metadata
databricks/bronze/ingest_raw_data.py - Bronze Ingestion → SQL Server Access: reads EHR CDC
databricks/bronze/ingest_raw_data.py - Pipeline Operator → FHIR Normalization: runs stage
README.md - Pipeline Operator → Relational Normalization: runs stage
README.md - FHIR Normalization → Silver Engine: delegates to
databricks/silver/process_fhir_silver.py - Relational Normalization → Silver Engine: delegates to
databricks/silver/process_relational_silver.py - Silver Engine → Silver Mapping Specs: processes specs
databricks/silver/silver_engine.py - Silver Engine → Bronze Reader: reads bronze
databricks/silver/bronze_reader.py - Silver Engine → Quality and Quarantine: validates batches
databricks/silver/silver_engine.py - Silver Engine → Silver Delta: merges silver
databricks/silver/silver_engine.py - Pipeline Operator → Dimension Builder: runs dimensions
README.md - Pipeline Operator → Fact Builder: runs facts
README.md - Dimension Builder → Gold Delta Warehouse: builds dimensions
README.md - Fact Builder → Gold Delta Warehouse: builds facts
README.md - Fact Builder → Silver Warehouse Reader: reads silver
databricks/gold/build_facts.py - Fact Builder → Warehouse Keys: uses key functions
databricks/gold/build_facts.py - Dimension Builder → Warehouse Keys: uses key functions
databricks/gold/build_dimensions.py - Analytics Consumer → Analytics Dashboards: queries analytics
README.md - Analytics Dashboards → Gold Delta Warehouse: queries warehouse
README.md - Bronze Ingestion → Run Audit: records attempts
README.md - Silver Engine → Run Audit: records runs
databricks/silver/silver_engine.py - Bronze Ingestion → Failure Alerts: raises alerts
databricks/bronze/ingest_raw_data.py - Silver Engine → Failure Alerts: raises alerts
databricks/silver/silver_engine.py - Failure Alerts → Alert Webhook: posts optionally
databricks/utilities/alerting.py - Pipeline Operator → Failure Recovery Demo: runs recovery demo
README.md - Failure Recovery Demo → Bronze Ingestion: replays ingestion
databricks/utilities/failure_simulation.py - Failure Recovery Demo → Silver Engine: replays processing
databricks/utilities/failure_simulation.py - Failure Recovery Demo → SQL Server Access: corrupts source
databricks/utilities/failure_simulation.py