How sharmilnk/from-air-to-care works
1. Repository purpose
- This project predicts hospital admissions for respiratory distress from environmental signals such as air quality, weather, and seasonality.
- Architecturally, it is split into a data science pipeline at the repo root and a small Flask-style web app under NYC_Health_App_Light_Mode/.
2. Primary data sources and stored datasets
- The data/ directory holds the persisted tabular inputs used by the pipeline:
- Air_Quality.csv
- Asthama.csv
- Respiratory.csv
- nyc_weather_by_borough_2017-2024.csv
- These files form the local data boundary between external public datasets and the model-training code.
3. Data ingestion / enrichment layer
- fetch_weather.py appears to handle retrieval of NYC weather data, likely building or updating the weather dataset from an external source.
- The README indicates the broader source set includes OpenAQ, EPA, NOAA, and NYC DOHMH, so this repo is designed around multi-source public-data ingestion rather than a single internal database.
4. Data preparation layer
- prepare_data.py is the main transformation step that likely joins environmental variables with health outcomes, creates lagged features, and produces analysis-ready tables.
- This is the key feature-engineering boundary in the system: raw CSV inputs in data/ become model-ready time-aligned records.
5. Exploration and analysis
- eda.py is the exploratory analysis module, and plots/ stores generated visual outputs.
- This subsystem supports understanding relationships between pollution, weather, and admissions before training, but is separate from the serving app.
6. Model training and evaluation
- main.py is the primary entry point for the modeling workflow.
- The README says several regressors were evaluated: Gradient Boosting, Random Forest, Lasso, and Ridge, with Gradient Boosting selected as the best model.
- This implies the training layer is a classical tabular ML pipeline rather than deep learning or a streaming system.
7. Application/serving subsystem
- NYC_Health_App_Light_Mode/app.py is the application entry point for a user-facing interface.
- rebuild_models.py suggests a separate utility for retraining or regenerating model artifacts without going through the full analysis pipeline manually.
- This creates a clear boundary between offline training and online prediction/UI.
8. Web UI structure
- The app uses a standard template/static layout:
- templates/index.html for the page structure
- static/css/style.css for styling
- static/js/script.js for client-side behavior
- This indicates a lightweight server-rendered interface rather than a SPA or API-only deployment.
9. Runtime and dependencies
- requirements.txt defines a Python-based stack.
- Based on the structure, the repo likely depends on common data-science libraries plus a lightweight web framework for the app layer.
- There is no sign of containerization, orchestration, or external service infrastructure in the tree.
10. End-to-end data flow
- External public datasets are collected or refreshed.
- Data is cleaned and merged into analysis/model tables.
- EDA and plotting validate assumptions and relationships.
- main.py trains and evaluates regression models.
- The selected model is then consumed by the app in NYC_Health_App_Light_Mode/ for a user-facing forecast experience.
- The main architectural split is therefore: offline environmental-health forecasting pipeline + lightweight presentation layer.
Main entry points: app.py Flask app · main.py Train model · prepare_data.py Prepare data · requirements.txt Dependencies
Components 14
Data boundary
Persisted inputs and intermediate files for the forecasting pipeline
- Public data · OpenAQ, EPA, NOAA, and NYC DOHMH inputs referenced by the project
- Data files · Persisted air quality, respiratory, asthma, and weather tables
data
Forecasting pipeline
Ingestion, preparation, analysis, and model training
- Fetch weather · Retrieves or refreshes NYC weather data
fetch_weather.py - Prepare data · Joins signals, aligns time series, and builds model-ready tables
prepare_data.py - EDA · Explores relationships among pollution, weather, and admissions
eda.py - Plots · Generated charts and exploratory visuals
plots - Train model · Runs regression experiments and selects the best predictor
main.py - Gradient boosting · Chosen tabular model for admissions prediction
Serving app
Lightweight web UI and model rebuild entry points
- Dependencies · Library set used by the pipeline and app
NYC_Health_App_Light_Mode/requirements.txt - Flask app · User-facing application entry point
NYC_Health_App_Light_Mode/app.py - Rebuild models · Utility for retraining or regenerating artifacts
NYC_Health_App_Light_Mode/rebuild_models.py - Page template · Server-rendered UI structure
NYC_Health_App_Light_Mode/templates/index.html - Stylesheet · Presentation styling for the web app
NYC_Health_App_Light_Mode/static/css/style.css - Client script · Browser-side behavior for the interface
NYC_Health_App_Light_Mode/static/js/script.js
Connections 14
- Public data → Fetch weather: feeds · Public datasets are ingested or refreshed
- Fetch weather → Data files: stores · Weather data is persisted alongside other inputs
- Data files → Prepare data: inputs · Raw tables are combined into analysis-ready records
- Prepare data → EDA: enables · Prepared data supports exploration and validation
- EDA → Plots: writes · EDA produces charts and diagnostic visuals
- Prepare data → Train model: features · Model training consumes engineered tables
- Train model → Gradient boosting: selects · Gradient boosting is chosen as the best regressor
- Gradient boosting → Flask app: served by · Selected model powers the user-facing app
- Dependencies → Flask app: supports · Shared Python dependencies underpin the app runtime
- Dependencies → Train model: supports · Shared Python dependencies underpin training
- Flask app → Page template: renders · Server-side rendering uses the page template
- Flask app → Stylesheet: uses · The UI is styled by static CSS
- Flask app → Client script: uses · Client-side behavior is provided by JavaScript
- Rebuild models → Train model: reuses · Rebuild utility regenerates model artifacts through the training workflow