How knownkey/imf-cpi-ppi-datapull works
1. What this repo is
- A small Python CLI exporter that pulls monthly CPI/PPI index data from two external statistical sources and writes a styled Excel workbook.
- It is not a service or UI app; the main boundary is command line in, .xlsx file out.
2. Core runtime and packaging
- The implementation is centered in app.py, with a lightweight dependency set in requirements.txt.
- The repo also includes a devcontainer for a reproducible Python development environment.
- The checked-in .xlsx file is a generated output artifact, not part of the runtime architecture.
3. Primary data sources
- IMF CPI raw monthly index data.
- IMF PPI raw monthly index data.
- BLS PPI series WPU00000000 for U.S. all commodities.
- These are the only external systems the app depends on at runtime.
4. Execution flow
- The CLI accepts optional filters such as:
- country list
- start period
- end period
- It then fetches observations from each source, transforms them into a common monthly structure, and exports the result to Excel.
- Default behavior is to export all available IMF countries plus U.S. BLS data.
5. Source-specific ingestion layer
- The architecture has three logical source adapters:
- IMF CPI
- IMF PPI
- BLS PPI
- Each adapter is responsible for pulling raw observations from its own API format and returning data in a shape the rest of the pipeline can normalize.
6. Transformation / normalization layer
- Raw observations are converted into a country-by-month matrix.
- Data is organized so that each country becomes a column and each month becomes a row entry under a year grouping.
- Period labels are rendered as human-readable months, and values are kept at consistent numeric precision.
7. Workbook generation layer
- The Excel output is split into three sheets:
- CPI Raw Data
- PPI Raw Data
- BLS PPI Raw Data
- Each sheet has the same structure:
- metadata rows at the top
- bold country-name header row
- ISO3 code summary row
- “Most Recent Data” row
- a left-side year column
- centered numeric data cells
8. Formatting as part of the architecture
- Styling is not cosmetic only; it is a deliberate output concern in the export pipeline.
- The workbook layout encodes provenance and recency directly in the sheet, so consumers can inspect data without needing the CLI or source APIs.
9. Output boundary
- The final artifact is a date-stamped Excel workbook, named by default with the current date, e.g. IMF_CPI_PPI_Raw_20260518.xlsx.
- The app writes to the local filesystem only; there is no database, object store, or downstream service.
10. Test architecture
- tests/test_app.py and tests/test_bls.py cover the main behaviors.
- The tests focus on:
- mocked data fetching
- pivot/transformation correctness
- sheet structure and layout
- Excel formatting for both IMF and BLS paths
- This indicates the repo treats the data contract and workbook shape as the critical interface.
11. Operational dependencies
- Runtime reliability depends on external API availability and response shape from IMF/BLS.
- Because the data is pulled live and written immediately, there is no persistent intermediate store or cache layer in the repository architecture.
12. High-level diagram view
- A useful architecture diagram would show:
- CLI entrypoint (app.py)
- three external data sources (IMF CPI, IMF PPI, BLS PPI)
- a shared normalization/merge step
- an Excel workbook renderer
- local .xlsx output
- tests validating the fetch/transform/render pipeline
Main entry points: app.py Workbook · test_app.py test_app.py · requirements.txt requirements.txt · test_bls.py test_bls.py
Components 18
Runtime
CLI execution and data pipeline
- app.py · Command-line entrypoint and main pipeline orchestration
app.py - CLI args · Country and date filters accepted by the exporter
app.py - Source router · Coordinates the three source-specific ingestion paths
app.py - Normalize · Converts raw observations into a common monthly structure
app.py - Monthly matrix · Organizes data into year rows and country columns
app.py - Period labels · Renders human-readable month labels and numeric precision
app.py
Sources
External statistical APIs
Output
Excel workbook generation and filesystem write
- Workbook · Creates the styled Excel workbook with three sheets
app.py - CPI Raw Data · Workbook sheet for IMF CPI data
app.py - PPI Raw Data · Workbook sheet for IMF PPI data
app.py - BLS PPI Raw Data · Workbook sheet for BLS PPI data
app.py - .xlsx file · Date-stamped Excel output written locally
Validation
Tests covering fetch, transform, and sheet layout
- test_app.py · Validates mocked fetching, transformation, and workbook structure
tests/test_app.py - test_bls.py · Validates BLS-specific ingestion and formatting behavior
tests/test_bls.py
Environment
Local Python setup and dependencies
- requirements.txt · Runtime dependency list for the exporter
requirements.txt - devcontainer · Reproducible local development environment
.devcontainer/devcontainer.json
Connections 20
- app.py → CLI args: parses · Reads optional country and date filters from the CLI
- app.py → Source router: orchestrates · Coordinates the end-to-end export run
- Source router → IMF CPI: fetches · Retrieves IMF CPI monthly observations
- Source router → IMF PPI: fetches · Retrieves IMF PPI monthly observations
- Source router → BLS PPI: fetches · Retrieves BLS PPI observations
- IMF CPI → Normalize: raw data · Provides source-shaped observations for normalization
- IMF PPI → Normalize: raw data · Provides source-shaped observations for normalization
- BLS PPI → Normalize: raw data · Provides source-shaped observations for normalization
- Normalize → Monthly matrix: pivots · Converts observations into a country-by-month matrix
- Monthly matrix → Period labels: formats · Adds month labels and consistent numeric display
- Period labels → Workbook: builds · Feeds the worksheet renderer with normalized rows
- Workbook → CPI Raw Data: sheet · Renders the IMF CPI worksheet
- Workbook → PPI Raw Data: sheet · Renders the IMF PPI worksheet
- Workbook → BLS PPI Raw Data: sheet · Renders the BLS PPI worksheet
- Workbook → .xlsx file: writes · Persists the final date-stamped workbook locally
- test_app.py → app.py: covers · Exercises the main fetch-transform-render path
- test_app.py → Workbook: checks · Verifies sheet structure and layout
- test_bls.py → BLS PPI: covers · Exercises BLS-specific ingestion behavior
- requirements.txt → app.py: supports · Declares runtime packages used by the exporter
- devcontainer → requirements.txt: provisions · Creates a reproducible Python environment