# yashsheth0107/civichack architecture

> Architecture diagram of the GitHub repository yashsheth0107/civichack, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yashsheth0107/civichack
- Repository: https://github.com/yashsheth0107/civichack
- Video tour (about a minute): https://gitdiagram.com/yashsheth0107/civichack/video

## Overview

Council Tracker is a resident-facing Toronto municipal decision dashboard backed by a data-ingestion, lifecycle-tracking, and vote-prediction pipeline. Its main workflow fetches the City’s report-request and voting datasets, joins follow-up items to recorded outcomes, classifies lifecycle stages, and presents searchable decisions. A separate vote-model branch engineers historical features, trains a pass-probability model, and turns predictions into close-call engagement priorities. Edges reflect README statements and indexed source references; unsampled dashboard and prediction details are represented only where documented.

Main entry points: [`app.py`](https://github.com/yashsheth0107/civichack/blob/main/app.py) (Streamlit dashboard), [`src/model/train_outcome_model.py`](https://github.com/yashsheth0107/civichack/blob/main/src/model/train_outcome_model.py) (Outcome trainer), [`src/ingest/fetch_datasets.py`](https://github.com/yashsheth0107/civichack/blob/main/src/ingest/fetch_datasets.py) (Dataset downloader), [`src/ingest/toronto_open_data.py`](https://github.com/yashsheth0107/civichack/blob/main/src/ingest/toronto_open_data.py) (CKAN client)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingest["Data ingestion"]
  node_config["Dataset configuration<br/>[config.py]"]
  node_fetch["Dataset downloader<br/>[fetch_datasets.py]"]
  node_ckan["CKAN client"]
  node_raw["Raw datasets"]
end

subgraph group_pipeline["Decision lifecycle"]
  node_request_loader["Request-log parser"]
  node_vote_loader["Voting-record parser"]
  node_join["Follow-up join<br/>[join_decisions.py]"]
  node_stage["Lifecycle classifier<br/>[stage.py]"]
end

subgraph group_model["Vote prediction"]
  node_features["Vote features<br/>[features.py]"]
  node_trainer["Outcome trainer"]
  node_predictor["Outcome predictor<br/>[predict.py]"]
  node_priority["Engagement scoring"]
  node_model_artifact["Trained model"]
end

subgraph group_presentation["Resident experience"]
  node_dashboard["Streamlit dashboard<br/>[app.py]"]
end

node_resident(("Resident"))
node_toronto(("Toronto Open Data"))

node_resident -->|"uses"| node_dashboard
node_dashboard -->|"displays stages"| node_stage
node_dashboard -->|"shows priorities"| node_priority
node_dashboard -->|"bootstraps data"| node_fetch
node_dashboard -->|"bootstraps model"| node_trainer
node_fetch -->|"reads settings"| node_config
node_fetch -->|"finds and downloads"| node_ckan
node_ckan -->|"uses endpoint"| node_config
node_ckan -->|"requests resources"| node_toronto
node_fetch -->|"saves files"| node_raw
node_join -->|"loads requests"| node_request_loader
node_join -->|"loads vote events"| node_vote_loader
node_stage -->|"builds decisions"| node_join
node_trainer -->|"loads vote events"| node_vote_loader
node_trainer -->|"builds features"| node_features
node_features -->|"uses event data"| node_vote_loader
node_trainer -->|"saves model"| node_model_artifact
node_predictor -->|"uses trained model"| node_trainer
node_predictor -->|"scores predictions"| node_priority
node_dashboard -->|"supports exploration"| node_predictor

click node_config "https://github.com/yashsheth0107/civichack/blob/main/config.py"
click node_fetch "https://github.com/yashsheth0107/civichack/blob/main/src/ingest/fetch_datasets.py"
click node_ckan "https://github.com/yashsheth0107/civichack/blob/main/src/ingest/toronto_open_data.py"
click node_raw "https://github.com/yashsheth0107/civichack/tree/main/data/raw"
click node_request_loader "https://github.com/yashsheth0107/civichack/blob/main/src/pipeline/load_report_request_log.py"
click node_vote_loader "https://github.com/yashsheth0107/civichack/blob/main/src/pipeline/load_voting_record.py"
click node_join "https://github.com/yashsheth0107/civichack/blob/main/src/pipeline/join_decisions.py"
click node_stage "https://github.com/yashsheth0107/civichack/blob/main/src/pipeline/stage.py"
click node_features "https://github.com/yashsheth0107/civichack/blob/main/src/model/features.py"
click node_trainer "https://github.com/yashsheth0107/civichack/blob/main/src/model/train_outcome_model.py"
click node_predictor "https://github.com/yashsheth0107/civichack/blob/main/src/model/predict.py"
click node_priority "https://github.com/yashsheth0107/civichack/blob/main/src/model/engagement_score.py"
click node_model_artifact "https://github.com/yashsheth0107/civichack/tree/main/models"
click node_dashboard "https://github.com/yashsheth0107/civichack/blob/main/app.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_config,node_fetch,node_ckan,node_raw toneBlue
class node_request_loader,node_vote_loader,node_join,node_stage toneAmber
class node_features,node_trainer,node_predictor,node_priority,node_model_artifact toneMint
class node_dashboard toneRose
class node_resident,node_toronto toneIndigo
```

## Components

### Data ingestion

- [Dataset configuration](https://github.com/yashsheth0107/civichack/blob/main/config.py): `config.py`
- [Dataset downloader](https://github.com/yashsheth0107/civichack/blob/main/src/ingest/fetch_datasets.py): `src/ingest/fetch_datasets.py`
- [CKAN client](https://github.com/yashsheth0107/civichack/blob/main/src/ingest/toronto_open_data.py): `src/ingest/toronto_open_data.py`
- [Raw datasets](https://github.com/yashsheth0107/civichack/tree/main/data/raw): `data/raw`

### Decision lifecycle

- [Request-log parser](https://github.com/yashsheth0107/civichack/blob/main/src/pipeline/load_report_request_log.py): `src/pipeline/load_report_request_log.py`
- [Voting-record parser](https://github.com/yashsheth0107/civichack/blob/main/src/pipeline/load_voting_record.py): `src/pipeline/load_voting_record.py`
- [Follow-up join](https://github.com/yashsheth0107/civichack/blob/main/src/pipeline/join_decisions.py): `src/pipeline/join_decisions.py`
- [Lifecycle classifier](https://github.com/yashsheth0107/civichack/blob/main/src/pipeline/stage.py): `src/pipeline/stage.py`

### Vote prediction

- [Vote features](https://github.com/yashsheth0107/civichack/blob/main/src/model/features.py): `src/model/features.py`
- [Outcome trainer](https://github.com/yashsheth0107/civichack/blob/main/src/model/train_outcome_model.py): `src/model/train_outcome_model.py`
- [Outcome predictor](https://github.com/yashsheth0107/civichack/blob/main/src/model/predict.py): `src/model/predict.py`
- [Engagement scoring](https://github.com/yashsheth0107/civichack/blob/main/src/model/engagement_score.py): `src/model/engagement_score.py`
- [Trained model](https://github.com/yashsheth0107/civichack/tree/main/models): `models`

### Resident experience

- [Streamlit dashboard](https://github.com/yashsheth0107/civichack/blob/main/app.py): `app.py`

### Other components

- **Resident**
- **Toronto Open Data**

## Connections

- Resident → Streamlit dashboard: uses (evidence: `README.md`)
- Streamlit dashboard → Lifecycle classifier: displays stages (evidence: `README.md`)
- Streamlit dashboard → Engagement scoring: shows priorities (evidence: `app.py`)
- Streamlit dashboard → Dataset downloader: bootstraps data (evidence: `app.py`)
- Streamlit dashboard → Outcome trainer: bootstraps model (evidence: `app.py`)
- Dataset downloader → Dataset configuration: reads settings (evidence: `src/ingest/fetch_datasets.py`)
- Dataset downloader → CKAN client: finds and downloads (evidence: `src/ingest/fetch_datasets.py`)
- CKAN client → Dataset configuration: uses endpoint (evidence: `src/ingest/toronto_open_data.py`)
- CKAN client → Toronto Open Data: requests resources (evidence: `src/ingest/toronto_open_data.py`)
- Dataset downloader → Raw datasets: saves files (evidence: `src/ingest/fetch_datasets.py`)
- Follow-up join → Request-log parser: loads requests (evidence: `src/pipeline/join_decisions.py`)
- Follow-up join → Voting-record parser: loads vote events (evidence: `src/pipeline/join_decisions.py`)
- Lifecycle classifier → Follow-up join: builds decisions (evidence: `src/pipeline/stage.py`)
- Outcome trainer → Voting-record parser: loads vote events (evidence: `src/model/train_outcome_model.py`)
- Outcome trainer → Vote features: builds features (evidence: `src/model/train_outcome_model.py`)
- Vote features → Voting-record parser: uses event data (evidence: `src/model/features.py`)
- Outcome trainer → Trained model: saves model (evidence: `README.md`)
- Outcome predictor → Outcome trainer: uses trained model (evidence: `src/model/predict.py`)
- Outcome predictor → Engagement scoring: scores predictions (evidence: `src/model/predict.py`)
- Streamlit dashboard → Outcome predictor: supports exploration (evidence: `README.md`)
