# jeevanshbhatia650-rgb/procasto_ai architecture

> Architecture diagram of the GitHub repository jeevanshbhatia650-rgb/procasto_ai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/jeevanshbhatia650-rgb/procasto_ai
- Repository: https://github.com/jeevanshbhatia650-rgb/procasto_ai

## Overview

PROCASTO-AI is a streaming smart-home assistant: speech or text becomes stable clauses, a revisioned plan starts live-state, manual, and conversation retrieval before the utterance ends, and cited answer cards can update as device state changes. Corrections can park plans and reuse fresh evidence on resume. The graph preserves the documented voice, simulator, SmartThings, Alexa, and confirmed-command capabilities. Some subsystem links use exact enclosing directories because their implementation was not sampled; those links reflect README-documented behavior.

Main entry points: [`backend/app/session/manager.py`](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/session/manager.py) (Session manager), [`frontend/src/lib/ws.ts`](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/frontend/src/lib/ws.ts) (WebSocket client), [`backend/app/session/pipeline.py`](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/session/pipeline.py) (Task pipeline), [`backend/app/api/ws.py`](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/api/ws.py) (WebSocket API)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interaction["User interaction"]
  node_frontend["Assistant UI<br/>[App.tsx]"]
  node_speech_input["Speech input<br/>[useSpeechInput.ts]"]
  node_ws_client["WebSocket client<br/>[ws.ts]"]
  node_answer_ui["Cards and timeline<br/>[AnswerCards.tsx]"]
  node_ws_api["WebSocket API<br/>[ws.py]"]
  node_alexa["Alexa endpoint<br/>[routes_alexa.py]"]
end

subgraph group_planning["Session and planning"]
  node_session["Session manager<br/>[manager.py]"]
  node_nlu["Clause extraction"]
  node_plan["Revisioned plans<br/>[query_plan.py]"]
  node_invalidation["Plan invalidation<br/>[invalidation.py]"]
  node_pipeline["Task pipeline<br/>[pipeline.py]"]
  node_orchestrator["Retrieval orchestrator<br/>[orchestrator.py]"]
end

subgraph group_knowledge["Retrieval and answers"]
  node_retrievers["Retrieval sources"]
  node_manuals["Manual search<br/>[manual_search.py]"]
  node_manual_ingest["Manual ingestion<br/>[manual_ingest.py]"]
  node_evidence["Evidence and precedence"]
  node_composer["Answer composer<br/>[composer.py]"]
  node_llm["Optional phrasing"]
end

subgraph group_devices["Device integrations"]
  node_live_store["Live state store<br/>[live_store.py]"]
  node_simulator["Home simulator<br/>[simulator.py]"]
  node_smartthings["SmartThings integration"]
  node_commands["Command gate<br/>[commands.py]"]
end

node_user(("User"))
node_alexa_service(("Alexa skill"))
node_smartthings_service(("SmartThings"))
node_manual_files["Manual files"]
node_llm_service(("Gemini API"))

node_user -->|"asks or corrects"| node_frontend
node_frontend -->|"captures speech"| node_speech_input
node_speech_input -->|"streams transcripts"| node_ws_client
node_frontend -->|"sends text"| node_ws_client
node_ws_client -->|"sends events"| node_ws_api
node_ws_api -->|"dispatches input"| node_session
node_session -->|"extracts clauses"| node_nlu
node_session -->|"updates plan"| node_plan
node_session -->|"applies plan diff"| node_pipeline
node_pipeline -->|"runs tasks"| node_orchestrator
node_orchestrator -->|"dispatches retrieval"| node_retrievers
node_retrievers -->|"reads state"| node_live_store
node_retrievers -->|"searches manuals"| node_manuals
node_manual_ingest -->|"indexes sections"| node_manuals
node_manual_files -->|"provides content"| node_manual_ingest
node_orchestrator -->|"stores results"| node_evidence
node_pipeline -->|"composes cards"| node_composer
node_composer -->|"resolves facts"| node_evidence
node_composer -.->|"requests phrasing"| node_llm
node_llm -.->|"phrases facts"| node_llm_service
node_pipeline -->|"publishes updates"| node_ws_api
node_ws_api -->|"streams updates"| node_ws_client
node_ws_client -->|"updates cards"| node_answer_ui
node_live_store -->|"notifies device changes"| node_session
node_session -->|"checks material changes"| node_invalidation
node_invalidation -->|"invalidates tasks"| node_plan
node_simulator -->|"emits state changes"| node_live_store
node_smartthings_service -.->|"sends signed events"| node_smartthings
node_smartthings -.->|"applies device events"| node_live_store
node_session -->|"executes confirmed command"| node_commands
node_commands -.->|"controls simulated device"| node_simulator
node_commands -.->|"controls real device"| node_smartthings
node_alexa_service -.->|"sends utterance"| node_alexa
node_alexa -.->|"answers request"| node_session

click node_frontend "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/frontend/src/App.tsx"
click node_speech_input "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/frontend/src/hooks/useSpeechInput.ts"
click node_ws_client "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/frontend/src/lib/ws.ts"
click node_answer_ui "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/frontend/src/components/user/AnswerCards.tsx"
click node_ws_api "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/api/ws.py"
click node_session "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/session/manager.py"
click node_nlu "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/nlu/clause_extractor.py"
click node_plan "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/planning/query_plan.py"
click node_invalidation "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/planning/invalidation.py"
click node_pipeline "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/session/pipeline.py"
click node_orchestrator "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/planning/orchestrator.py"
click node_retrievers "https://github.com/jeevanshbhatia650-rgb/procasto_ai/tree/feat/procasto-ai/backend/app/retrieval"
click node_manuals "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/retrieval/manual_search.py"
click node_manual_ingest "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/retrieval/manual_ingest.py"
click node_evidence "https://github.com/jeevanshbhatia650-rgb/procasto_ai/tree/feat/procasto-ai/backend/app/evidence"
click node_composer "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/answer/composer.py"
click node_llm "https://github.com/jeevanshbhatia650-rgb/procasto_ai/tree/feat/procasto-ai/backend/app/llm"
click node_live_store "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/state/live_store.py"
click node_simulator "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/devices/simulator.py"
click node_smartthings "https://github.com/jeevanshbhatia650-rgb/procasto_ai/tree/feat/procasto-ai/backend/app/devices/smartthings"
click node_commands "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/devices/commands.py"
click node_alexa "https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/api/routes_alexa.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend,node_speech_input,node_ws_client,node_answer_ui,node_ws_api,node_alexa,node_user toneBlue
class node_session,node_nlu,node_plan,node_invalidation,node_pipeline,node_orchestrator toneAmber
class node_retrievers,node_manuals,node_manual_ingest,node_evidence,node_composer,node_llm,node_llm_service toneMint
class node_live_store,node_simulator,node_smartthings,node_commands toneRose
class node_alexa_service,node_smartthings_service,node_manual_files toneIndigo
```

## Components

### User interaction

- [Assistant UI](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/frontend/src/App.tsx): `frontend/src/App.tsx`
- [Speech input](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/frontend/src/hooks/useSpeechInput.ts): `frontend/src/hooks/useSpeechInput.ts`
- [WebSocket client](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/frontend/src/lib/ws.ts): `frontend/src/lib/ws.ts`
- [Cards and timeline](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/frontend/src/components/user/AnswerCards.tsx): `frontend/src/components/user/AnswerCards.tsx`
- [WebSocket API](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/api/ws.py): `backend/app/api/ws.py`
- [Alexa endpoint](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/api/routes_alexa.py): `backend/app/api/routes_alexa.py`

### Session and planning

- [Session manager](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/session/manager.py): `backend/app/session/manager.py`
- [Clause extraction](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/nlu/clause_extractor.py): `backend/app/nlu/clause_extractor.py`
- [Revisioned plans](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/planning/query_plan.py): `backend/app/planning/query_plan.py`
- [Plan invalidation](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/planning/invalidation.py): `backend/app/planning/invalidation.py`
- [Task pipeline](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/session/pipeline.py): `backend/app/session/pipeline.py`
- [Retrieval orchestrator](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/planning/orchestrator.py): `backend/app/planning/orchestrator.py`

### Retrieval and answers

- [Retrieval sources](https://github.com/jeevanshbhatia650-rgb/procasto_ai/tree/feat/procasto-ai/backend/app/retrieval): `backend/app/retrieval`
- [Manual search](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/retrieval/manual_search.py): `backend/app/retrieval/manual_search.py`
- [Manual ingestion](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/retrieval/manual_ingest.py): `backend/app/retrieval/manual_ingest.py`
- [Evidence and precedence](https://github.com/jeevanshbhatia650-rgb/procasto_ai/tree/feat/procasto-ai/backend/app/evidence): `backend/app/evidence`
- [Answer composer](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/answer/composer.py): `backend/app/answer/composer.py`
- [Optional phrasing](https://github.com/jeevanshbhatia650-rgb/procasto_ai/tree/feat/procasto-ai/backend/app/llm): `backend/app/llm`

### Device integrations

- [Live state store](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/state/live_store.py): `backend/app/state/live_store.py`
- [Home simulator](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/devices/simulator.py): `backend/app/devices/simulator.py`
- [SmartThings integration](https://github.com/jeevanshbhatia650-rgb/procasto_ai/tree/feat/procasto-ai/backend/app/devices/smartthings): `backend/app/devices/smartthings`
- [Command gate](https://github.com/jeevanshbhatia650-rgb/procasto_ai/blob/feat/procasto-ai/backend/app/devices/commands.py): `backend/app/devices/commands.py`

### Other components

- **User**
- **Alexa skill**
- **SmartThings**
- **Manual files**
- **Gemini API**

## Connections

- User → Assistant UI: asks or corrects
- Assistant UI → Speech input: captures speech
- Speech input → WebSocket client: streams transcripts
- Assistant UI → WebSocket client: sends text
- WebSocket client → WebSocket API: sends events
- WebSocket API → Session manager: dispatches input
- Session manager → Clause extraction: extracts clauses
- Session manager → Revisioned plans: updates plan
- Session manager → Task pipeline: applies plan diff
- Task pipeline → Retrieval orchestrator: runs tasks
- Retrieval orchestrator → Retrieval sources: dispatches retrieval
- Retrieval sources → Live state store: reads state
- Retrieval sources → Manual search: searches manuals
- Manual ingestion → Manual search: indexes sections
- Manual files → Manual ingestion: provides content
- Retrieval orchestrator → Evidence and precedence: stores results
- Task pipeline → Answer composer: composes cards
- Answer composer → Evidence and precedence: resolves facts
- Answer composer → Optional phrasing: requests phrasing
- Optional phrasing → Gemini API: phrases facts
- Task pipeline → WebSocket API: publishes updates
- WebSocket API → WebSocket client: streams updates
- WebSocket client → Cards and timeline: updates cards
- Live state store → Session manager: notifies device changes
- Session manager → Plan invalidation: checks material changes
- Plan invalidation → Revisioned plans: invalidates tasks
- Home simulator → Live state store: emits state changes
- SmartThings → SmartThings integration: sends signed events
- SmartThings integration → Live state store: applies device events
- Session manager → Command gate: executes confirmed command
- Command gate → Home simulator: controls simulated device
- Command gate → SmartThings integration: controls real device
- Alexa skill → Alexa endpoint: sends utterance
- Alexa endpoint → Session manager: answers request