# knownkey/imf-cpi-ppi-datapull architecture

> Architecture diagram of the GitHub repository knownkey/imf-cpi-ppi-datapull, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-05-23). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/knownkey/imf-cpi-ppi-datapull
- Repository: https://github.com/knownkey/imf-cpi-ppi-datapull

## Overview

1. **What this repo is**
   - A small **Python CLI exporter** that pulls monthly CPI/PPI index data from two external statistical sources and writes a **styled Excel workbook**.
   - It is not a service or UI app; the main boundary is **command line in, `.xlsx` file out**.

2. **Core runtime and packaging**
   - The implementation is centered in **`app.py`**, with a lightweight dependency set in **`requirements.txt`**.
   - The repo also includes a **devcontainer** for a reproducible Python development environment.
   - The checked-in `.xlsx` file is a generated output artifact, not part of the runtime architecture.

3. **Primary data sources**
   - **IMF CPI** raw monthly index data.
   - **IMF PPI** raw monthly index data.
   - **BLS PPI** series `WPU00000000` for U.S. all commodities.
   - These are the only external systems the app depends on at runtime.

4. **Execution flow**
   - The CLI accepts optional filters such as:
     - country list
     - start period
     - end period
   - It then fetches observations from each source, transforms them into a common monthly structure, and exports the result to Excel.
   - Default behavior is to export all available IMF countries plus U.S. BLS data.

5. **Source-specific ingestion layer**
   - The architecture has three logical source adapters:
     - IMF CPI
     - IMF PPI
     - BLS PPI
   - Each adapter is responsible for pulling raw observations from its own API format and returning data in a shape the rest of the pipeline can normalize.

6. **Transformation / normalization layer**
   - Raw observations are converted into a **country-by-month matrix**.
   - Data is organized so that each country becomes a column and each month becomes a row entry under a year grouping.
   - Period labels are rendered as human-readable months, and values are kept at consistent numeric precision.

7. **Workbook generation layer**
   - The Excel output is split into **three sheets**:
     - `CPI Raw Data`
     - `PPI Raw Data`
     - `BLS PPI Raw Data`
   - Each sheet has the same structure:
     - metadata rows at the top
     - bold country-name header row
     - ISO3 code summary row
     - “Most Recent Data” row
     - a left-side year column
     - centered numeric data cells

8. **Formatting as part of the architecture**
   - Styling is not cosmetic only; it is a deliberate output concern in the export pipeline.
   - The workbook layout encodes provenance and recency directly in the sheet, so consumers can inspect data without needing the CLI or source APIs.

9. **Output boundary**
   - The final artifact is a **date-stamped Excel workbook**, named by default with the current date, e.g. `IMF_CPI_PPI_Raw_20260518.xlsx`.
   - The app writes to the local filesystem only; there is no database, object store, or downstream service.

10. **Test architecture**
   - `tests/test_app.py` and `tests/test_bls.py` cover the main behaviors.
   - The tests focus on:
     - mocked data fetching
     - pivot/transformation correctness
     - sheet structure and layout
     - Excel formatting for both IMF and BLS paths
   - This indicates the repo treats the **data contract and workbook shape** as the critical interface.

11. **Operational dependencies**
   - Runtime reliability depends on external API availability and response shape from IMF/BLS.
   - Because the data is pulled live and written immediately, there is no persistent intermediate store or cache layer in the repository architecture.

12. **High-level diagram view**
   - A useful architecture diagram would show:
     - CLI entrypoint (`app.py`)
     - three external data sources (IMF CPI, IMF PPI, BLS PPI)
     - a shared normalization/merge step
     - an Excel workbook renderer
     - local `.xlsx` output
     - tests validating the fetch/transform/render pipeline

Main entry points: [`app.py`](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py) (Workbook), [`tests/test_app.py`](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/tests/test_app.py) (test_app.py), [`requirements.txt`](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/requirements.txt) (requirements.txt), [`tests/test_bls.py`](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/tests/test_bls.py) (test_bls.py)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Runtime"]
  node_app["app.py<br/>cli exporter<br/>[app.py]"]
  node_cli_args["CLI args<br/>filters<br/>[app.py]"]
  node_source_router["Source router<br/>dispatch<br/>[app.py]"]
  node_normalize["Normalize<br/>transform<br/>[app.py]"]
  node_pivot_matrix["Monthly matrix<br/>pivot<br/>[app.py]"]
  node_render_labels["Period labels<br/>formatting<br/>[app.py]"]
end

subgraph group_sources["Sources"]
  node_imf_cpi["IMF CPI<br/>adapter<br/>[app.py]"]
  node_imf_ppi["IMF PPI<br/>adapter<br/>[app.py]"]
  node_bls_ppi["BLS PPI<br/>adapter<br/>[app.py]"]
end

subgraph group_output["Output"]
  node_workbook["Workbook<br/>xlsx builder<br/>[app.py]"]
  node_cpi_sheet["CPI Raw Data<br/>worksheet<br/>[app.py]"]
  node_ppi_sheet["PPI Raw Data<br/>worksheet<br/>[app.py]"]
  node_bls_sheet["BLS PPI Raw Data<br/>worksheet<br/>[app.py]"]
  node_xlsx_file[".xlsx file<br/>filesystem artifact"]
end

subgraph group_validation["Validation"]
  node_tests_app["test_app.py<br/>pipeline tests<br/>[test_app.py]"]
  node_tests_bls["test_bls.py<br/>source tests<br/>[test_bls.py]"]
end

subgraph group_environment["Environment"]
  node_requirements["requirements.txt<br/>deps<br/>[requirements.txt]"]
  node_devcontainer["devcontainer<br/>python env<br/>[devcontainer.json]"]
end

node_app -->|"parses"| node_cli_args
node_app -->|"orchestrates"| node_source_router
node_source_router -->|"fetches"| node_imf_cpi
node_source_router -->|"fetches"| node_imf_ppi
node_source_router -->|"fetches"| node_bls_ppi
node_imf_cpi -->|"raw data"| node_normalize
node_imf_ppi -->|"raw data"| node_normalize
node_bls_ppi -->|"raw data"| node_normalize
node_normalize -->|"pivots"| node_pivot_matrix
node_pivot_matrix -->|"formats"| node_render_labels
node_render_labels -->|"builds"| node_workbook
node_workbook -->|"sheet"| node_cpi_sheet
node_workbook -->|"sheet"| node_ppi_sheet
node_workbook -->|"sheet"| node_bls_sheet
node_workbook -->|"writes"| node_xlsx_file
node_tests_app -.->|"covers"| node_app
node_tests_app -.->|"checks"| node_workbook
node_tests_bls -.->|"covers"| node_bls_ppi
node_requirements -.->|"supports"| node_app
node_devcontainer -.->|"provisions"| node_requirements

click node_app "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_cli_args "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_source_router "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_imf_cpi "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_imf_ppi "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_bls_ppi "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_normalize "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_pivot_matrix "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_render_labels "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_workbook "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_cpi_sheet "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_ppi_sheet "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_bls_sheet "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py"
click node_tests_app "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/tests/test_app.py"
click node_tests_bls "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/tests/test_bls.py"
click node_requirements "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/requirements.txt"
click node_devcontainer "https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/.devcontainer/devcontainer.json"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_cli_args,node_source_router,node_normalize,node_pivot_matrix,node_render_labels toneBlue
class node_imf_cpi,node_imf_ppi,node_bls_ppi toneAmber
class node_workbook,node_cpi_sheet,node_ppi_sheet,node_bls_sheet,node_xlsx_file toneMint
class node_tests_app,node_tests_bls toneRose
class node_requirements,node_devcontainer toneIndigo
```

## Components

### Runtime

CLI execution and data pipeline

- [app.py](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): cli exporter · Command-line entrypoint and main pipeline orchestration · `app.py`
- [CLI args](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): filters · Country and date filters accepted by the exporter · `app.py`
- [Source router](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): dispatch · Coordinates the three source-specific ingestion paths · `app.py`
- [Normalize](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): transform · Converts raw observations into a common monthly structure · `app.py`
- [Monthly matrix](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): pivot · Organizes data into year rows and country columns · `app.py`
- [Period labels](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): formatting · Renders human-readable month labels and numeric precision · `app.py`

### Sources

External statistical APIs

- [IMF CPI](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): adapter · Fetches monthly CPI observations from IMF in source-specific format · `app.py`
- [IMF PPI](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): adapter · Fetches monthly PPI observations from IMF in source-specific format · `app.py`
- [BLS PPI](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): adapter · Fetches U.S. all-commodities PPI series from BLS · `app.py`

### Output

Excel workbook generation and filesystem write

- [Workbook](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): xlsx builder · Creates the styled Excel workbook with three sheets · `app.py`
- [CPI Raw Data](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): worksheet · Workbook sheet for IMF CPI data · `app.py`
- [PPI Raw Data](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): worksheet · Workbook sheet for IMF PPI data · `app.py`
- [BLS PPI Raw Data](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/app.py): worksheet · Workbook sheet for BLS PPI data · `app.py`
- **.xlsx file**: filesystem artifact · Date-stamped Excel output written locally

### Validation

Tests covering fetch, transform, and sheet layout

- [test_app.py](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/tests/test_app.py): pipeline tests · Validates mocked fetching, transformation, and workbook structure · `tests/test_app.py`
- [test_bls.py](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/tests/test_bls.py): source tests · Validates BLS-specific ingestion and formatting behavior · `tests/test_bls.py`

### Environment

Local Python setup and dependencies

- [requirements.txt](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/requirements.txt): deps · Runtime dependency list for the exporter · `requirements.txt`
- [devcontainer](https://github.com/knownkey/imf-cpi-ppi-datapull/blob/main/.devcontainer/devcontainer.json): python env · Reproducible local development environment · `.devcontainer/devcontainer.json`

## Connections

- app.py → CLI args: parses · Reads optional country and date filters from the CLI
- app.py → Source router: orchestrates · Coordinates the end-to-end export run
- Source router → IMF CPI: fetches · Retrieves IMF CPI monthly observations
- Source router → IMF PPI: fetches · Retrieves IMF PPI monthly observations
- Source router → BLS PPI: fetches · Retrieves BLS PPI observations
- IMF CPI → Normalize: raw data · Provides source-shaped observations for normalization
- IMF PPI → Normalize: raw data · Provides source-shaped observations for normalization
- BLS PPI → Normalize: raw data · Provides source-shaped observations for normalization
- Normalize → Monthly matrix: pivots · Converts observations into a country-by-month matrix
- Monthly matrix → Period labels: formats · Adds month labels and consistent numeric display
- Period labels → Workbook: builds · Feeds the worksheet renderer with normalized rows
- Workbook → CPI Raw Data: sheet · Renders the IMF CPI worksheet
- Workbook → PPI Raw Data: sheet · Renders the IMF PPI worksheet
- Workbook → BLS PPI Raw Data: sheet · Renders the BLS PPI worksheet
- Workbook → .xlsx file: writes · Persists the final date-stamped workbook locally
- test_app.py → app.py: covers · Exercises the main fetch-transform-render path
- test_app.py → Workbook: checks · Verifies sheet structure and layout
- test_bls.py → BLS PPI: covers · Exercises BLS-specific ingestion behavior
- requirements.txt → app.py: supports · Declares runtime packages used by the exporter
- devcontainer → requirements.txt: provisions · Creates a reproducible Python environment
