# dylantirandaz/forecast architecture

> Architecture diagram of the GitHub repository dylantirandaz/forecast, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dylantirandaz/forecast
- Repository: https://github.com/dylantirandaz/forecast

## Overview

Forecast is an AI-assisted forecasting platform for NYC housing-policy outcomes, with a FastAPI backend, Celery workers, persistent data, and a React interface. The main workflow is question/scenario input, historical-prior and evidence analysis, forecast generation, then evaluation and calibration. Separate backtest, benchmark, experiment, replay, and cost features support model assessment. The graph also includes the standalone `forecast/` LLM-evaluation toolkit. Some UI-to-API and unsampled subsystem connections are not visible in the excerpts, so those links are omitted or marked unverified.

Main entry points: [`backend/app/api/evaluations.py`](https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/evaluations.py) (Evaluation endpoints), [`backend/app/services/replay_engine.py`](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/replay_engine.py) (Replay engine), [`backend/app/main.py`](https://github.com/dylantirandaz/forecast/blob/main/backend/app/main.py) (FastAPI app), [`backend/app/api/router.py`](https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/router.py) (API router)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["User interface"]
  node_web["Forecast dashboard<br/>[page.tsx]"]
  node_questions_ui["Question views<br/>[page.tsx]"]
  node_eval_ui["Evaluation views<br/>[page.tsx]"]
end

subgraph group_api["API and orchestration"]
  node_api_app["FastAPI app<br/>[main.py]"]
  node_api_router["API router<br/>[router.py]"]
  node_question_api["Question endpoints<br/>[questions.py]"]
  node_forecast_api["Forecast endpoints<br/>[forecasts.py]"]
  node_scenario_api["Scenario endpoints<br/>[scenarios.py]"]
  node_common_schema["Common schemas<br/>[common.py]"]
  node_task_worker["Forecast tasks<br/>[forecast_tasks.py]"]
end

subgraph group_forecast["Forecasting domain"]
  node_forecast_engine["Forecast engine<br/>[forecast_engine.py]"]
  node_base_rates["Base-rate engine"]
  node_evidence["Evidence scoring<br/>[evidence_scorer.py]"]
  node_belief["Belief updating<br/>[belief_updater.py]"]
  node_scenario_engine["Scenario engine<br/>[scenario_engine.py]"]
end

subgraph group_assessment["Evaluation and learning"]
  node_eval_api["Evaluation endpoints<br/>[evaluations.py]"]
  node_replay["Replay engine<br/>[replay_engine.py]"]
  node_metrics["Evaluation metrics<br/>[eval_metrics.py]"]
  node_backtester["Backtesting<br/>[backtester.py]"]
  node_calibration["Calibration<br/>[calibration.py]"]
  node_benchmark["Benchmark harness"]
  node_experiments["Experiment tracking"]
  node_ablations["Ablation runner<br/>[ablation_runner.py]"]
  node_costs["Cost tracking<br/>[cost_tracker.py]"]
end

subgraph group_data["Data and persistence"]
  node_db_session["DB sessions<br/>[database.py]"]
  node_database[("PostgreSQL")]
  node_adapters["Data adapters"]
end

subgraph group_toolkit["LLM evaluation toolkit"]
  node_forecast_config["Toolkit config<br/>[config.py]"]
  node_llm_client["LLM client<br/>[llm.py]"]
  node_prompts["Forecast prompts<br/>[prompts.py]"]
  node_search["Evidence search<br/>[search.py]"]
  node_tool_metrics["Forecast metrics<br/>[metrics.py]"]
end

node_user(("Forecaster"))
node_llm_provider{{"LLM provider"}}

node_user -->|"uses"| node_web
node_web -.->|"requests data"| node_api_app
node_questions_ui -.->|"requests data"| node_api_app
node_eval_ui -.->|"requests data"| node_api_app
node_api_app -->|"mounts routes"| node_api_router
node_api_router -->|"registers"| node_question_api
node_api_router -->|"registers"| node_forecast_api
node_api_router -->|"registers"| node_scenario_api
node_api_router -->|"registers"| node_eval_api
node_api_app -->|"checks connection"| node_db_session
node_db_session -->|"opens sessions"| node_database
node_api_app -->|"validates health"| node_common_schema
node_task_worker -->|"runs forecast"| node_forecast_engine
node_task_worker -->|"runs backtest"| node_backtester
node_task_worker -->|"dispatches ingestion"| node_adapters
node_forecast_engine -->|"computes prior"| node_base_rates
node_forecast_engine -->|"scores evidence"| node_evidence
node_forecast_engine -->|"updates belief"| node_belief
node_forecast_engine -->|"compares scenarios"| node_scenario_engine
node_eval_api -->|"runs replay"| node_replay
node_eval_api -->|"computes metrics"| node_metrics
node_eval_api -->|"uses session"| node_db_session
node_replay -.->|"scores evidence"| node_evidence
node_replay -.->|"updates belief"| node_belief
node_replay -.->|"computes prior"| node_base_rates
node_replay -.->|"uses calibration"| node_calibration
node_replay -.->|"tracks cost"| node_costs
node_forecast_config -->|"sets model costs"| node_llm_client
node_search -->|"reads quality settings"| node_forecast_config
node_search -->|"calls for review"| node_llm_client
node_search -->|"uses prompts"| node_prompts
node_llm_client -->|"sends completion"| node_llm_provider
node_experiments -->|"runs ablations"| node_ablations
node_eval_api -.->|"exposes evaluation"| node_benchmark
node_eval_api -.->|"exposes backtests"| node_backtester
node_eval_api -.->|"exposes calibration"| node_calibration
node_tool_metrics -.->|"evaluates forecasts"| node_llm_provider

click node_web "https://github.com/dylantirandaz/forecast/blob/main/frontend/src/app/page.tsx"
click node_questions_ui "https://github.com/dylantirandaz/forecast/blob/main/frontend/src/app/questions/page.tsx"
click node_eval_ui "https://github.com/dylantirandaz/forecast/blob/main/frontend/src/app/eval/page.tsx"
click node_api_app "https://github.com/dylantirandaz/forecast/blob/main/backend/app/main.py"
click node_api_router "https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/router.py"
click node_question_api "https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/questions.py"
click node_forecast_api "https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/forecasts.py"
click node_scenario_api "https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/scenarios.py"
click node_eval_api "https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/evaluations.py"
click node_db_session "https://github.com/dylantirandaz/forecast/blob/main/backend/app/database.py"
click node_common_schema "https://github.com/dylantirandaz/forecast/blob/main/backend/app/schemas/common.py"
click node_task_worker "https://github.com/dylantirandaz/forecast/blob/main/backend/app/tasks/forecast_tasks.py"
click node_adapters "https://github.com/dylantirandaz/forecast/tree/main/backend/app/adapters"
click node_forecast_engine "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/forecast_engine.py"
click node_base_rates "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/base_rate_engine.py"
click node_evidence "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/evidence_scorer.py"
click node_belief "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/belief_updater.py"
click node_scenario_engine "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/scenario_engine.py"
click node_replay "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/replay_engine.py"
click node_metrics "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/eval_metrics.py"
click node_backtester "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/backtester.py"
click node_calibration "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/calibration.py"
click node_benchmark "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/benchmark_harness.py"
click node_experiments "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/experiment_tracker.py"
click node_ablations "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/ablation_runner.py"
click node_costs "https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/cost_tracker.py"
click node_forecast_config "https://github.com/dylantirandaz/forecast/blob/main/forecast/config.py"
click node_llm_client "https://github.com/dylantirandaz/forecast/blob/main/forecast/llm.py"
click node_prompts "https://github.com/dylantirandaz/forecast/blob/main/forecast/prompts.py"
click node_search "https://github.com/dylantirandaz/forecast/blob/main/forecast/search.py"
click node_tool_metrics "https://github.com/dylantirandaz/forecast/blob/main/forecast/metrics.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_web,node_questions_ui,node_eval_ui toneBlue
class node_api_app,node_api_router,node_question_api,node_forecast_api,node_scenario_api,node_common_schema,node_task_worker toneAmber
class node_forecast_engine,node_base_rates,node_evidence,node_belief,node_scenario_engine toneMint
class node_eval_api,node_replay,node_metrics,node_backtester,node_calibration,node_benchmark,node_experiments,node_ablations,node_costs toneRose
class node_db_session,node_database,node_adapters,node_user,node_llm_provider toneIndigo
class node_forecast_config,node_llm_client,node_prompts,node_search,node_tool_metrics toneTeal
```

## Components

### User interface

- [Forecast dashboard](https://github.com/dylantirandaz/forecast/blob/main/frontend/src/app/page.tsx): `frontend/src/app/page.tsx`
- [Question views](https://github.com/dylantirandaz/forecast/blob/main/frontend/src/app/questions/page.tsx): `frontend/src/app/questions/page.tsx`
- [Evaluation views](https://github.com/dylantirandaz/forecast/blob/main/frontend/src/app/eval/page.tsx): `frontend/src/app/eval/page.tsx`

### API and orchestration

- [FastAPI app](https://github.com/dylantirandaz/forecast/blob/main/backend/app/main.py): `backend/app/main.py`
- [API router](https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/router.py): `backend/app/api/router.py`
- [Question endpoints](https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/questions.py): `backend/app/api/questions.py`
- [Forecast endpoints](https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/forecasts.py): `backend/app/api/forecasts.py`
- [Scenario endpoints](https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/scenarios.py): `backend/app/api/scenarios.py`
- [Common schemas](https://github.com/dylantirandaz/forecast/blob/main/backend/app/schemas/common.py): `backend/app/schemas/common.py`
- [Forecast tasks](https://github.com/dylantirandaz/forecast/blob/main/backend/app/tasks/forecast_tasks.py): `backend/app/tasks/forecast_tasks.py`

### Forecasting domain

- [Forecast engine](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/forecast_engine.py): `backend/app/services/forecast_engine.py`
- [Base-rate engine](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/base_rate_engine.py): `backend/app/services/base_rate_engine.py`
- [Evidence scoring](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/evidence_scorer.py): `backend/app/services/evidence_scorer.py`
- [Belief updating](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/belief_updater.py): `backend/app/services/belief_updater.py`
- [Scenario engine](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/scenario_engine.py): `backend/app/services/scenario_engine.py`

### Evaluation and learning

- [Evaluation endpoints](https://github.com/dylantirandaz/forecast/blob/main/backend/app/api/evaluations.py): `backend/app/api/evaluations.py`
- [Replay engine](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/replay_engine.py): `backend/app/services/replay_engine.py`
- [Evaluation metrics](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/eval_metrics.py): `backend/app/services/eval_metrics.py`
- [Backtesting](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/backtester.py): `backend/app/services/backtester.py`
- [Calibration](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/calibration.py): `backend/app/services/calibration.py`
- [Benchmark harness](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/benchmark_harness.py): `backend/app/services/benchmark_harness.py`
- [Experiment tracking](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/experiment_tracker.py): `backend/app/services/experiment_tracker.py`
- [Ablation runner](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/ablation_runner.py): `backend/app/services/ablation_runner.py`
- [Cost tracking](https://github.com/dylantirandaz/forecast/blob/main/backend/app/services/cost_tracker.py): `backend/app/services/cost_tracker.py`

### Data and persistence

- [DB sessions](https://github.com/dylantirandaz/forecast/blob/main/backend/app/database.py): `backend/app/database.py`
- **PostgreSQL**
- [Data adapters](https://github.com/dylantirandaz/forecast/tree/main/backend/app/adapters): `backend/app/adapters`

### LLM evaluation toolkit

- [Toolkit config](https://github.com/dylantirandaz/forecast/blob/main/forecast/config.py): `forecast/config.py`
- [LLM client](https://github.com/dylantirandaz/forecast/blob/main/forecast/llm.py): `forecast/llm.py`
- [Forecast prompts](https://github.com/dylantirandaz/forecast/blob/main/forecast/prompts.py): `forecast/prompts.py`
- [Evidence search](https://github.com/dylantirandaz/forecast/blob/main/forecast/search.py): `forecast/search.py`
- [Forecast metrics](https://github.com/dylantirandaz/forecast/blob/main/forecast/metrics.py): `forecast/metrics.py`

### Other components

- **Forecaster**
- **LLM provider**

## Connections

- Forecaster → Forecast dashboard: uses (evidence: `README.md`)
- Forecast dashboard → FastAPI app: requests data
- Question views → FastAPI app: requests data
- Evaluation views → FastAPI app: requests data
- FastAPI app → API router: mounts routes (evidence: `backend/app/main.py`)
- API router → Question endpoints: registers (evidence: `backend/app/api/router.py`)
- API router → Forecast endpoints: registers (evidence: `backend/app/api/router.py`)
- API router → Scenario endpoints: registers (evidence: `backend/app/api/router.py`)
- API router → Evaluation endpoints: registers (evidence: `backend/app/api/router.py`)
- FastAPI app → DB sessions: checks connection (evidence: `backend/app/main.py`)
- DB sessions → PostgreSQL: opens sessions (evidence: `backend/app/database.py`)
- FastAPI app → Common schemas: validates health (evidence: `backend/app/main.py`)
- Forecast tasks → Forecast engine: runs forecast (evidence: `backend/app/tasks/forecast_tasks.py`)
- Forecast tasks → Backtesting: runs backtest (evidence: `backend/app/tasks/forecast_tasks.py`)
- Forecast tasks → Data adapters: dispatches ingestion (evidence: `backend/app/tasks/forecast_tasks.py`)
- Forecast engine → Base-rate engine: computes prior (evidence: `backend/app/services/forecast_engine.py`)
- Forecast engine → Evidence scoring: scores evidence (evidence: `backend/app/services/forecast_engine.py`)
- Forecast engine → Belief updating: updates belief (evidence: `backend/app/services/forecast_engine.py`)
- Forecast engine → Scenario engine: compares scenarios (evidence: `backend/app/services/forecast_engine.py`)
- Evaluation endpoints → Replay engine: runs replay (evidence: `backend/app/api/evaluations.py`)
- Evaluation endpoints → Evaluation metrics: computes metrics (evidence: `backend/app/api/evaluations.py`)
- Evaluation endpoints → DB sessions: uses session (evidence: `backend/app/api/evaluations.py`)
- Replay engine → Evidence scoring: scores evidence
- Replay engine → Belief updating: updates belief
- Replay engine → Base-rate engine: computes prior
- Replay engine → Calibration: uses calibration
- Replay engine → Cost tracking: tracks cost
- Toolkit config → LLM client: sets model costs (evidence: `forecast/llm.py`)
- Evidence search → Toolkit config: reads quality settings (evidence: `forecast/search.py`)
- Evidence search → LLM client: calls for review (evidence: `forecast/search.py`)
- Evidence search → Forecast prompts: uses prompts (evidence: `forecast/search.py`)
- LLM client → LLM provider: sends completion (evidence: `forecast/llm.py`)
- Experiment tracking → Ablation runner: runs ablations (evidence: `backend/app/api/experiments.py`)
- Evaluation endpoints → Benchmark harness: exposes evaluation
- Evaluation endpoints → Backtesting: exposes backtests
- Evaluation endpoints → Calibration: exposes calibration
- Forecast metrics → LLM provider: evaluates forecasts
