How sfbrigade/datasci-firerisk works
Project type and purpose
• This is a data-science–driven predictive modeling and visualization project (“San Francisco Fire Risk Project”) that ingests heterogeneous data sources, builds a machine-learning model to estimate relative fire risk, and produces an interactive map/dashboard as the user-facing interface.
• Its primary features include data ingestion/cleaning, feature engineering (address matching, distance to station, tax data aggregation, complaint features), model training & scoring, and map generation for visualization.
File structure
• Top-level scripts & notebooks:
• process.py, final_dataset_upload.py, column_report.py, deduplicate_model_output.py, multiclass.py (batch-oriented ETL, training, reporting)
• Jupyter notebooks (.ipynb) and R scripts for exploratory data analysis (EDA)
• GIS utility: gis.py
• map_template.html / map.html for final visualization
• package.json + lib/*.js for JS-based merging, aggregation, prediction tasks
• Data directories:
• addresses/ (raw address ZIP)
• address_matching/ (Python module to match addresses)
• nearest_fire_station/ (distance calculation to stations, Google API wrappers)
• External config/build: .gitignore, .idea, README.md
• No monolithic web-server; visualization is generated as static HTML (CartoDB embeds, Leaflet, etc.), but could be wrapped in a simple Node/Express host if desired.
Extract architecture insights from README
• Data sources: SF OpenData APIs, external Google Distance API, shapefiles, tax & complaint data
• Tech stack: Python (NumPy, pandas, scikit-learn, matplotlib), R (dplyr, ggplot2), Jupyter, JavaScript (for light ETL), CartoDB for map hosting
Final reminders
• Be exhaustive: represent each directory or script as a component or sub-component.
• Batch vs. interactive pathways.
• Config or parameter files if they affect the flow.