covid19india/data

Purpose and Published Surface

This repository is a batch data-publishing pipeline for COVID-19 India statistics, not an application server. It transforms Google Sheet-derived/raw inputs into static CSV datasets and V4 JSON feeds served at data.covid19india.org. The active consumer surface is V4 (v4/min/data.min.json, v4/min/timeseries.min.json) plus aggregated CSV; legacy JSON endpoints are deprecated.

Pipeline Orchestration

main.sh is the top-level batch entry point, sequencing generators and validation. The implementation spans Node.js and Python, with dependencies declared in package.json, package-lock.json, and requirements.txt. There is no persistent database or API runtime represented in the tree; generated static files are the delivery boundary.

Paths: main.sh, package.json, requirements.txt

Raw-Data Acquisition and Normalization

The pipeline consumes latest Google Sheet data and historical raw sheets, then converts sheet-shaped data to JSON/CSV representations. Generic sheet conversion and raw-data assembly form the input boundary; raw datasets preserve granular records and sources but are explicitly not the preferred downstream interface.

Paths: src/sheet-to-json_generic.js, src/sheets-to-csv.js, src/build_raw_data.py

State and District Aggregation

Dedicated transforms derive state daily series and district-level outputs from normalized/raw inputs. These stages establish the main geographic aggregation boundary used by CSV consumers and downstream V4 assembly.

Paths: src/states_daily_to_csv.js, src/district_data_generator.js, src/sanity_check.js

V4 API Data Construction

The V4 parser produces the structured JSON model used by the frontend: current state/district statistics and historical state time series. This is the primary product path; minified V4 files are the recommended externally consumed JSON artifacts.

Paths: src/parser_v4.py, lib/constants.js, documentation/v4_data.md

Supplemental Derived Datasets

Separate generators handle locale metadata and activity/change logs. These are adjacent published artifacts rather than core case aggregation, and likely consume the same generated/normalized data domain.

Paths: src/generate_locale.js, src/generate_activity_log.js, src/generate_activity_log.py

Validation and Operational Notification

Sanity checking is an explicit pre-publication quality gate for generated data. Telegram notification support indicates batch-job operational reporting; its invocation is likely coordinated through the shell workflow or Node entry module.

Paths: src/sanity_check.js, lib/notify_tg.js, lib/index.js

Infrastructure and External Boundaries

External dependencies are Google Sheets as the near-real-time source (noted as 10–20 minutes delayed), public static hosting at data.covid19india.org, and Telegram for notifications. GitHub Actions provides CI for JavaScript via .github/workflows/javascript.yml; no deployment workflow is visible. Documentation defines the public CSV, raw-data, and V4 contracts.

Paths: .github/workflows/javascript.yml, documentation/csv/index.md, documentation/rawdata.md