How bayareametro/bayarea_urbansim works
Project type and purpose
• This is a domain-specific, open-source simulation and data-processing tool built on the UrbanSim framework (Bay Area UrbanSim, “BAUS”).
• It is a command-line Python application that ingests large spatial and tabular data sets, runs a multi-stage urban development simulation, generates summaries and metrics, and optionally publishes notifications (Slack/Asana) and a Tableau-based visualizer.
File structure
• Top-level entrypoint:
• baus.py
• run_setup*.yaml files (model run configurations)
• Core Python package (baus/):
• datasources.py (loads raw inputs)
• preprocessing.py (data cleaning & feature engineering)
• models.py, slr.py, earthquake.py, subsidies.py, ual.py (simulation sub-models)
• summaries/ (post-model summary functions by theme)
• slack.py, debug.py, utils.py, variables.py
• visualizer/ (Tableau workbook, scripts to push artifacts)
• Configuration (configs/):
• Nested YAML/CSV files by module (accessibility, hedonic pricing, location choice, adjusters, transition relocation, developer settings, etc.)
• Scripts (scripts/):
• Pre- & post-processing helpers, validation checks, capacity analysis, metrics calculators, data exports, notebook debugging, external forces analyses, strategy-building notebooks and utilities, JSON/HTTP serving (serve_json.py)
• Documentation (docs/, mkdocs.yml) and tests (baus/tests/)
Key insights from README
• Invocation: python baus.py --run-setup run_setup.yaml [--disable-slack] [--enable-asana]
• Requires a Conda environment (baus-env-2023.yml)
• Optional integrations: Slack SDK, Asana SDK
• Optional Model Run Visualizer published to Tableau
• Command-Line Interface (baus.py + CLI args)
• Configuration Loader (YAML: run_setup + configs/)
• Data Sources layer (raw inputs from MTC servers or local folders)
• Preprocessing module
• Simulation Engine with sub-models (location choice, developer, hedonic price, subsidies, SLR, earthquake impacts, etc.)
• Summaries & Metrics module
• Output Writers (CSV/GeoJSON/Hyper files, logs)
• Integrations:
– Slack notifications
– Asana task updates
– Tableau Visualizer (push_model_files.py → Tableau workbook)
• Supporting scripts & services (serve_json for real-time JSON queries)
b. Relationships & interactions:
• CLI → Configuration Loader
• Configuration → Data Sources + Preprocessing
• Preprocessing → Simulation Engine
• Simulation Engine → Summaries/ Metrics
• Summaries → Output Writers (files, DB)
• Output Writers → Integrations (Slack/Asana/Visualizer)
• Supporting scripts may consume outputs or invoke sub-modules directly
c. Architectural patterns & principles:
• Modular design: each simulation concern in its own module
• Config-driven: behavior controlled by YAML files
• Separation of concerns: data access, model logic, summarization, notification, visualization
• Extensibility: add new summaries, adjusters, or sub-models by dropping in YAML + Python files
d. Technologies & frameworks:
• Python 3.x, UrbanSim library
• Pandas, NumPy, HDF5 or SQL (as configured)
• YAML for configuration
• Slack SDK, Asana SDK
• Tableau (.twb/.twbx), serve_json (Flask or similar)