How jmyers911mm/ns11mm-data-platform works
This is a Snowflake dbt data platform for museum reporting and analytics. The principal workflow ingests source data into immutable RAW tables, transforms it through staging and intermediate models into mart dimensions, facts, and reports, then serves analytics through semantic views and reporting tools. The README documents the live DPR and report-estate domains, ticket-demand forecasting, and semantic views; other listed integrations and models include planned or disabled scope. Pipeline calls are evidenced by sampled source; broader model and consumer links rely on the README’s architecture description.
Main entry points: snowflake_client.py Snowflake landing · gateway/pipeline.py Gateway ticketing · counterpoint/pipeline.py CounterPoint POS · raw RAW tables
Components 19
Source ingestion
- Gateway ticketing
pipelines/gateway/pipeline.py - CounterPoint POS
pipelines/counterpoint/pipeline.py - Shopify exports
pipelines/shopify/pipeline.py - Other integrations
pipelines
Warehouse modeling
- RAW tables
models/raw - Staging models
models/raw - Intermediate models
models/intermediate - Mart dimensions
models/marts/dimensions - Mart facts
models/marts/facts - Mart reports
models/marts/reports - Forecast features
models/ml_features
Analytics serving
- Semantic views
cortex_project - DPR dashboard
dpr-dashboard/streamlit_app.py
Data quality
- Quality checks
macros/data_quality
Pipeline utilities
- Snowflake landing
pipelines/shared/snowflake_client.py - Key Vault secrets
pipelines/shared/keyvault.py - Refresh logging
pipelines/shared/refresh_log.py
Other components
- Analysts and users
- Power BI and Snowsight
Connections 22
- Gateway ticketing → Snowflake landing: lands records
pipelines/gateway/pipeline.py - CounterPoint POS → Snowflake landing: lands records
pipelines/counterpoint/pipeline.py - Shopify exports → Snowflake landing: lands records
pipelines/shopify/pipeline.py - Other integrations → Snowflake landing: lands records
pipelines/salesforce_nps/pipeline.py - Gateway ticketing → Key Vault secrets: fetches secrets
pipelines/gateway/pipeline.py - CounterPoint POS → Key Vault secrets: fetches secrets
pipelines/counterpoint/pipeline.py - Shopify exports → Key Vault secrets: fetches secrets
pipelines/shopify/pipeline.py - Gateway ticketing → Refresh logging: writes run log
pipelines/gateway/pipeline.py - CounterPoint POS → Refresh logging: writes run log
pipelines/counterpoint/pipeline.py - Snowflake landing → RAW tables: writes append-only
README.md - RAW tables → Staging models: transforms
README.md - Staging models → Intermediate models: transforms
README.md - Intermediate models → Mart dimensions: builds marts
README.md - Intermediate models → Mart facts: builds marts
README.md - Mart dimensions → Mart reports: serves reporting
README.md - Mart facts → Mart reports: serves reporting
README.md - Mart facts → Forecast features: feeds features
README.md - Mart reports → Semantic views: serves analytics
README.md - Semantic views → Power BI and Snowsight: serves views
README.md - Semantic views → Analysts and users: supports analysis
README.md - Analysts and users → DPR dashboard: uses dashboard
- RAW tables → Quality checks: checks data