# usgpo/bill-status architecture

> Architecture diagram of the GitHub repository usgpo/bill-status, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/usgpo/bill-status
- Repository: https://github.com/usgpo/bill-status

## Overview

##### Project Type and Purpose
- This is neither a traditional “frontend-backend” app nor a compiler; it’s a data-pipeline + static-content repository that harvests, organizes, and publishes U.S. Congress Bill Status XML in bulk.
- It runs scheduled jobs to pull XML files from the GPO/GovInfo Bulk Data API, organizes them (by Congress, by sample category), generates ancillary artifacts (RSS feed, sitemap), and publishes everything via static hosting (GovInfo’s site + GitHub).

a. Scheduler/Orchestrator
– Cron or workflow engine triggering two pipelines:
- “Current Congress” job (every 4 hrs)
- “Prior Congress” job (daily)
b. Ingestion Service
– Scripts (Python/Perl/Node/etc.) that call the GPO Bulk Data REST endpoints, fetch XML bundles for specified ranges.
– Handles retries, error logging.
c. Processing & Organization
– Post-fetch “ETL”: unzip/process XML, move into directory structure (samples/actions-committees, amendment-votes, formatChange/current|future, legacy, textVersions).
– Any filtering or enrichment (e.g., extracting sample files).
d. Storage Layer
– File system or cloud object store (S3/GCS) holding:
- All raw and organized XML files
- Generated RSS feed XML
- Sitemap XML
e. Publication Layer
– Static hosting (GitHub Pages + GovInfo Bulk Data site) serving:
- /bulkdata/BILLSTATUS/… directories
- RSS feed endpoint (/rss/billstatus-batch.xml)
– Automatic sitemap index (/sitemap/bulkdata/BILLSTATUS/…)
f. Documentation Artifacts
– User guides (PDF/MD), CHANGELOG, meeting slides—published alongside or linked from the site.

- Scheduler → Ingestion Service
- Ingestion Service → Processing/Organization
- Processing/Organization → Storage Layer
- Storage Layer → Publication Layer (trigger static site rebuild or sync)
- Storage Layer → RSS Generator → Storage Layer (write updated feed)
- Publication Layer → End users (via GovInfo website, GitHub, RSS clients)

##### Architectural Patterns & Principles
- Batch-Oriented ETL Pipeline (discrete runs on schedule)
- Static-Site Publication (no dynamic backend)
- Clear separation between data ingestion, processing, storage, and publication
- Use of existing GovInfo Bulk Data APIs and static-hosting infrastructure

##### Relevant Technologies & Frameworks
- Cron or workflow scheduler (e.g., Jenkins, Airflow, plain cron)
- HTTP client libraries in Python/Node/etc. for REST calls
- File I/O and directory management
- Static site tooling (Jekyll, Hugo, or GovInfo’s own pipeline)
- XML generators for RSS and sitemap (simple templating)
- Git + GitHub Pages for hosting / versioning
- GovInfo CDN / GovInfo.gov bulk-data infrastructure

##### Essential Elements
- Clear component labels and boundaries
- Compute/Jobs
- Data Storage
- Publication/Consumer Endpoints
- Notes on timing (4 hrs vs. daily) and external dependencies (GPO Bulk Data API)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Batch Pipeline
    subgraph "Batch Pipeline"
        Scheduler["Scheduler/Orchestrator<br/><i>every 4hrs (current) / daily (prior)</i>"]:::compute
        Ingestion["Ingestion Service<br/>(fetch GPO Bulk API)"]:::compute
        Processing["Processing & Organization<br/>(ETL & sample organization)"]:::compute
    end

    %% Storage Layer
    subgraph "Storage Layer"
        XMLStore(("XML File Store")):::storage
        RSSGen["RSS Feed Generator"]:::compute
        RSSStore(("RSS Feed Store")):::storage
        subgraph "Sample Data Organization"
            SamplesAC["actions-committees"]:::storage
            SamplesAV["amendment-votes"]:::storage
            SamplesFCc["formatChange/current"]:::storage
            SamplesFCf["formatChange/future"]:::storage
            SamplesLG["legacy"]:::storage
            SamplesTV["textVersions"]:::storage
        end
    end

    %% Publication Layer
    subgraph "Publication Layer"
        StaticSite["Static Site Hosting<br/>(GitHub Pages + GovInfo)"]:::delivery
        RSSFeed["RSS Feed Endpoint"]:::delivery
        Sitemap["Sitemap Index"]:::delivery
    end

    %% Documentation Artifacts
    subgraph "Documentation Artifacts"
        UserGuidePDF["BILLSTATUS-XML_User-Guide-v1.pdf"]:::docs
        UserGuideMD["BILLSTATUS-XML_User_User-Guide.md"]:::docs
        CHANGELOG["CHANGELOG.md"]:::docs
        Meetings["meetings/"]:::docs
        README["README.md"]:::docs
        CODEOWNERS["CODEOWNERS"]:::docs
    end

    EndUsers>End Users / RSS Clients / Browsers]:::external

    %% Data Flows
    Scheduler -->|triggers| Ingestion
    Ingestion -->|fetch XML bundles| Processing
    Processing -->|organize & enrich| XMLStore
    XMLStore -->|trigger site sync| StaticSite
    XMLStore -->|"generate RSS"| RSSGen
    RSSGen -->|write feed| RSSStore
    XMLStore -->|generate sitemap| Sitemap
    RSSStore -->|serve feed| RSSFeed
    StaticSite -->|serve static content| EndUsers
    RSSFeed -->|provide feed| EndUsers
    Sitemap -->|provide index| EndUsers

    %% Click Events
    click UserGuidePDF "https://github.com/usgpo/bill-status/blob/main/BILLSTATUS-XML_User-Guide-v1.pdf"
    click UserGuideMD "https://github.com/usgpo/bill-status/blob/main/BILLSTATUS-XML_User_User-Guide.md"
    click CHANGELOG "https://github.com/usgpo/bill-status/blob/main/CHANGELOG.md"
    click Meetings "https://github.com/usgpo/bill-status/tree/main/meetings/"
    click SamplesAC "https://github.com/usgpo/bill-status/tree/main/samples/actions-committees/"
    click SamplesAV "https://github.com/usgpo/bill-status/tree/main/samples/amendment-votes/"
    click SamplesFCc "https://github.com/usgpo/bill-status/tree/main/samples/formatChange/current/"
    click SamplesFCf "https://github.com/usgpo/bill-status/tree/main/samples/formatChange/future/"
    click SamplesLG "https://github.com/usgpo/bill-status/tree/main/samples/legacy/"
    click SamplesTV "https://github.com/usgpo/bill-status/tree/main/samples/textVersions/"
    click README "https://github.com/usgpo/bill-status/blob/main/README.md"
    click CODEOWNERS "https://github.com/usgpo/bill-status/tree/main/CODEOWNERS"

    %% Styles
    classDef compute fill:#D0E8FF,stroke:#0366d6,color:#0366d6,stroke-width:1px;
    classDef storage fill:#E8F5E9,stroke:#2E7D32,color:#2E7D32,stroke-width:1px;
    classDef delivery fill:#FFF3E0,stroke:#EF6C00,color:#EF6C00,stroke-width:1px;
    classDef docs fill:#F3E5F5,stroke:#6A1B9A,color:#6A1B9A,stroke-width:1px;
    classDef external fill:#ECEFF1,stroke:#546E7A,color:#546E7A,stroke-width:1px;
```
