# cdcgov/ephtracking architecture

> Architecture diagram of the GitHub repository cdcgov/ephtracking, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cdcgov/ephtracking
- Repository: https://github.com/cdcgov/ephtracking

## Overview

##### Project Type and Purpose
- This is an open-source “method and code” repository rather than a single deployable application.
- It houses:
- Data-validation protocols (R, SAS scripts)
- Dataset-creation guides (XSD schemas, PDFs)
- Code examples (Python API calls)
- Tracking-method documentation
- Its goal: define and share CDC’s end-to-end data-ingestion, validation, standardization, and publishing pipeline for environmental public-health measures.

##### File Structure
- Top-level folders map to pipeline stages and supporting materials:
- Data Validation
- Measure and Dataset Creation
- Tracking Data Methods
- Coding Examples (client/API examples)
- Data Explorer (map layers, points of interest)
- Configuration files (e.g., .env-template) and scripts (Python, SAS, R) live alongside protocol PDFs and XSD schema files.
- No single “frontend” or “backend” folder; this is a methods repo, not the actual web-app source.

##### Extract Architectural Insights from README
- The Tracking Network consists of:
1. Data ingestion from national/state/local sources
2. Validation & standardization
3. Measure calculation & XML generation
4. Data publishing to a repository & API
5. Web-based visualization (Data Explorer, dashboards)
- The repo provides the *methods* and *tools* used in steps (2) and (3); front-end and API code live elsewhere.

##### Main Components & Relationships
a. Data Sources
- National, state, local public-health databases, environmental sensors
b. ETL & Validation Layer
- Data Validation Protocol (PDF)
- SAS & R scripts (folder: Data Validation/Sample Validation Scripts)
c. Schema & Measure Creation
- XSD schema files (Measure and Dataset Creation/Schema Files)
- Style guides & calculation methods (PDFs)
- XML Generator Tool (external repo)
d. Standardized Data Repository
- Centralized data store (assumed database or file server)
e. Public API Layer
- API that serves standardized measures (external repo)
f. Visualization Layer
- National Data Explorer & Dashboards (external system)
g. Developer Toolkit & Documentation
- Coding Examples (Python API usage, .env templates)
- Tracking Data Methods scripts & docs
- Subcounty project resources

– Data flow: Sources → Validation → Schema mapping & measure calc → Repository → API → Explorer/Dashboards
– Supporting docs and example code sit alongside each pipeline stage.

##### Architectural Patterns & Design Principles
- Modular ETL pipeline: each stage encapsulated (validation, transformation, schema enforcement)
- Separation of concerns: methods repo (this project) vs. runtime systems (API, front end)
- Schema-driven design: XSD files drive data structure and enforce standards
- Extensibility: open-source scripts and docs allow external partners to plug in

## Diagram (Mermaid)

```mermaid
graph TB
    %% Data Sources
    subgraph "Data Sources"
        DataSources["National/State/Local Sources & Sensors"]:::external
    end

    %% ETL & Validation Layer
    subgraph "ETL & Validation Layer"
        DVProtocol["Data Validation Protocol"]:::pipeline
        Scripts["Validation Scripts (SAS/R)"]:::pipeline
    end

    %% Schema & Measure Creation
    subgraph "Schema & Measure Creation"
        StyleGuide["Indicator & Measure Style Guide"]:::pipeline
        XSD["XSD Schemas"]:::pipeline
        Subcounty["Subcounty Data Guides"]:::pipeline
        XMLGen["XML Generator Tool"]:::external
    end

    %% Standardized Data Repository
    Repository["Standardized Data Repository"]:::pipeline

    %% Public API Layer
    PublicAPI["Public API (External)"]:::external

    %% Visualization Layer
    Visualization["National Data Explorer & Dashboards"]:::external

    %% Developer Toolkit & Documentation
    subgraph "Developer Toolkit & Documentation"
        CodingExamples["API Example Python & .env Templates"]:::docs
        TrackingMethods["Tracking Method Scripts & Docs"]:::docs
        WorkgroupDocs["Workgroup Project Documents"]:::docs
    end

    %% Data Flow
    DataSources -->|ingestion| DVProtocol
    DVProtocol -->|validation| Scripts
    Scripts -->|standardization| StyleGuide
    StyleGuide --> XSD
    XSD --> Subcounty
    Subcounty --> XMLGen
    XMLGen --> Repository
    Repository --> PublicAPI
    PublicAPI --> Visualization

    %% Support links
    CodingExamples -->|supports| DVProtocol
    CodingExamples -->|supports| Scripts
    TrackingMethods -->|provides| Scripts
    WorkgroupDocs -->|references| StyleGuide

    %% Click Events
    click DVProtocol "https://github.com/cdcgov/ephtracking/blob/master/Data Validation/Data_Validation_Protocol.pdf"
    click Scripts "https://github.com/cdcgov/ephtracking/blob/master/Data Validation/Sample Validation Scripts/*.sas and *.R"
    click StyleGuide "https://github.com/cdcgov/ephtracking/blob/master/Measure and Dataset Creation/IndicatorMeasureNamesStyleGuide_July2022.pdf"
    click XSD "https://github.com/cdcgov/ephtracking/blob/master/Measure and Dataset Creation/Schema Files/*.xsd"
    click Subcounty "https://github.com/cdcgov/ephtracking/blob/master/Measure and Dataset Creation/Subcounty Data/*.pdf"
    click Visualization "https://github.com/cdcgov/ephtracking/tree/master/Data Explorer/EPHTN_DE_*"
    click CodingExamples "https://github.com/cdcgov/ephtracking/tree/master/Coding Examples/API Example Python"
    click TrackingMethods "https://github.com/cdcgov/ephtracking/blob/master/Tracking Data Methods/AAR Example SAS Code.sas"
    click WorkgroupDocs "https://github.com/cdcgov/ephtracking/blob/master/Workgroup Projects/Fact Sheets/*.pdf"

    %% Styles
    classDef pipeline fill:#cce5ff,stroke:#333,stroke-width:1px
    classDef external fill:#d4edda,stroke:#333,stroke-width:1px
    classDef docs fill:#e2e3e5,stroke:#333,stroke-width:1px
```
