# satcherinstitute/health-equity-tracker architecture

> Architecture diagram of the GitHub repository satcherinstitute/health-equity-tracker, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/satcherinstitute/health-equity-tracker
- Repository: https://github.com/satcherinstitute/health-equity-tracker

## Overview

##### Project type and purpose
- This is a full-stack, data-centric web application built around a microservice pipeline. Its goal is to ingest, process, store, and expose health equity data by demographic groups and geographies, then render interactive visualizations in a React frontend.

##### File structure
- Top-level services:
- frontend – React/Vite application
- frontend_server – Node.js server that hosts the built React app and proxies API calls
- data_server – Python microservice serving exported JSON data files
- exporter – Python microservice that reads from BigQuery and writes JSON files to Cloud Storage
- run_gcs_to_bq – Python microservice ingesting/caching raw source data to GCS and loading it into BigQuery
- run_ingestion – older Python microservice for legacy datasources
- Shared code and pipeline orchestration:
- python/ – ingestion modules and datasources
- config/ – Terraform scripts provisioning GCP resources (Cloud Run, Pub/Sub, buckets, BigQuery)
- .github/workflows/ – GitHub Actions DAGs that trigger ingestion/load/export pipelines
- Data assets:
- data/ – manually managed “landing” data files
- e2e_tests/, frontend/playwright-tests – end-to-end test suites
- exporter, data_server, run_* have Dockerfiles and requirements specifying microservice dependencies.

##### README for insights
- The README outlines a 4-layer architecture:
1) Data ingestion (run_ingestion & run_gcs_to_bq) → BigQuery
2) Exporter → JSON in GCS buckets
3) Data API (data_server) → serves those JSON files
4) Frontend (frontend + frontend_server) → React UI
- It highlights Terraform for infra provisioning, GitHub Actions for CI/CD, and Netlify for PR previews.

##### Essential architectural elements
- How code changes trigger CI/CD and infra updates
- Data transform and storage boundaries (GCS, BigQuery)
- Clearly demarcate frontend vs backend, and data plane vs control plane.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Sources
    ext["External Data Sources\n(CDC APIs, CSVs, GraphQL)"]:::ingestion

    %% Ingestion Layer
    subgraph "Ingestion Layer"
        run_ing["run_ingestion\n(Cloud Run)"]:::ingestion
        run_gcs["run_gcs_to_bq\n(Cloud Run)"]:::ingestion
        data_manual["Landing Data Files\ndata/"]:::ingestion
        python_lib["Shared Ingestion Modules\npython/"]:::ingestion
    end

    %% Storage Layer
    subgraph "Storage Layer"
        gcs_landing["GCS Landing Buckets"]:::storage
        bigquery["BigQuery\n(datasets)"]:::storage
    end

    %% Export & API Layer
    subgraph "Export & API Layer"
        exporter["exporter\n(Cloud Run)"]:::export
        gcs_export["GCS Export Buckets"]:::export
        data_api["data_server\n(Cloud Run API)"]:::export
    end

    %% Presentation Layer
    subgraph "Presentation Layer"
        frontend_srv["frontend_server\n(Node.js)"]:::presentation
        frontend_app["React App\n(frontend/)"]:::presentation
        browser["Browser\n(User UI)"]:::presentation
    end

    %% CI/CD & Infra
    subgraph "CI/CD & Infra"
        gha["GitHub Actions\n(.github/workflows)"]:::infra
        terraform["Terraform\n(config/)"]:::infra
        devs["Developers"]:::infra
    end

    %% Data Flows
    ext --> run_ing
    run_ing --> gcs_landing
    data_manual --> gcs_landing
    gcs_landing --> run_gcs
    python_lib --> run_ing
    python_lib --> run_gcs
    run_gcs --> bigquery
    bigquery --> exporter
    exporter --> gcs_export
    gcs_export --> data_api
    data_api --> frontend_srv
    frontend_srv --> frontend_app
    frontend_app --> browser
    browser --> data_api

    %% CI/CD Flows
    devs -.-> gha
    gha --> terraform
    terraform --> run_ing
    terraform --> run_gcs
    terraform --> exporter
    terraform --> data_api
    terraform --> frontend_srv

    %% Click Events
    click run_ing "https://github.com/satcherinstitute/health-equity-tracker/blob/main/run_ingestion/main.py"
    click run_gcs "https://github.com/satcherinstitute/health-equity-tracker/blob/main/run_gcs_to_bq/main.py"
    click exporter "https://github.com/satcherinstitute/health-equity-tracker/blob/main/exporter/main.py"
    click data_api "https://github.com/satcherinstitute/health-equity-tracker/blob/main/data_server/main.py"
    click frontend_app "https://github.com/satcherinstitute/health-equity-tracker/tree/main/frontend/"
    click frontend_srv "https://github.com/satcherinstitute/health-equity-tracker/tree/main/frontend_server/"
    click python_lib "https://github.com/satcherinstitute/health-equity-tracker/tree/main/python/"
    click gha "https://github.com/satcherinstitute/health-equity-tracker/tree/main/.github/workflows/"
    click terraform "https://github.com/satcherinstitute/health-equity-tracker/blob/main/config/main.tf"

    %% Styles
    classDef ingestion fill:#D0E8FF,stroke:#4DA1FF,color:#000;
    classDef storage fill:#DFFFE0,stroke:#3FB64F,color:#000;
    classDef export fill:#FFE8B0,stroke:#FFA800,color:#000;
    classDef presentation fill:#F3E0FF,stroke:#A64AC9,color:#000;
    classDef infra fill:#E0E0E0,stroke:#888,color:#000;
    class devs,gha,terraform infra
    class ext ingestion
    class run_ing,run_gcs,data_manual,python_lib ingestion
    class gcs_landing,bigquery storage
    class exporter,gcs_export,data_api export
    class frontend_srv,frontend_app,browser presentation
```
