How owid/covid-19-data works
This is OWID’s archived COVID-19 data-production repository. Its principal workflow is to collect and process COVID-related datasets, then generate internal and public outputs; the README distinguishes these production scripts from the final datasets consumed by users. The tree shows separate pipelines for cases and deaths, testing, hospitalizations, variants, vaccinations, and excess mortality, alongside shared cleaning and export code. Several domain modules were not sampled, so their internal calls and precise orchestration are not asserted; edges below rely on README statements or the source index.
Main entry points: base.py Vaccination processing · generate.py Dataset generator · clean Data cleaning · testing Testing data
Components 20
Dataset production
- Production scripts
- Dataset generator
scripts/src/cowidev/megafile/generate.py - Dataset steps
scripts/src/cowidev/megafile/steps - Dataset exports
scripts/src/cowidev/megafile/export - Public datasets
Data domains
- Cases and deaths
scripts/src/cowidev/cases_deaths.py - Testing data
scripts/src/cowidev/testing - Hospital data
scripts/src/cowidev/hosp/etl.py - Variant data
scripts/src/cowidev/variants/etl.py - Excess mortality
scripts/src/cowidev/xm/etl.py
Vaccination data
- Vaccination collection
scripts/src/cowidev/vax/batch/ecdc.py - Vaccination processing
scripts/src/cowidev/vax/utils/base.py - Vaccination files
scripts/src/cowidev/vax/utils/files.py - Manual social sources
scripts/src/cowidev/vax/manual/twitter/base.py
Shared services
- Path configuration
scripts/src/cowidev/utils/paths.py - Data cleaning
scripts/src/cowidev/utils/clean - Web downloads
scripts/src/cowidev/utils/web/download.py - S3 storage
scripts/src/cowidev/utils/s3.py - Population enrichment
scripts/scripts/shared.py
Other components
- Data maintainer
Connections 14
- Data maintainer → Production scripts: runs pipelines
README.md - Production scripts → Dataset generator: generates dataset
- Dataset generator → Dataset steps: loads steps
scripts/src/cowidev/megafile/steps/__init__.py - Dataset generator → Dataset exports: uses exports
scripts/src/cowidev/megafile/export/__init__.py - Dataset exports → Public datasets: writes datasets
- Vaccination processing → S3 storage: loads data
scripts/src/cowidev/vax/utils/base.py - Vaccination processing → Vaccination files: uses files
scripts/src/cowidev/vax/utils/base.py - Vaccination processing → Path configuration: reads paths
scripts/src/cowidev/vax/utils/base.py - Vaccination processing → Data cleaning: cleans dates
scripts/src/cowidev/vax/utils/base.py - Vaccination collection → Vaccination processing: uses processing
scripts/src/cowidev/vax/batch/ecdc.py - Manual social sources → Data cleaning: cleans metrics
scripts/src/cowidev/vax/manual/twitter/paraguay.py - Testing data → Data cleaning: cleans values
scripts/src/cowidev/testing/incremental/nicaragua.py - Testing data → Web downloads: downloads sources
scripts/src/cowidev/testing/incremental/nicaragua.py - Population enrichment → Public datasets: enriches measures