How jeevanshbhatia650-rgb/procasto_ai works
PROCASTO-AI is a streaming smart-home assistant: speech or text becomes stable clauses, a revisioned plan starts live-state, manual, and conversation retrieval before the utterance ends, and cited answer cards can update as device state changes. Corrections can park plans and reuse fresh evidence on resume. The graph preserves the documented voice, simulator, SmartThings, Alexa, and confirmed-command capabilities. Some subsystem links use exact enclosing directories because their implementation was not sampled; those links reflect README-documented behavior.
Main entry points: manager.py Session manager · ws.ts WebSocket client · pipeline.py Task pipeline · ws.py WebSocket API
Components 27
User interaction
- Assistant UI
frontend/src/App.tsx - Speech input
frontend/src/hooks/useSpeechInput.ts - WebSocket client
frontend/src/lib/ws.ts - Cards and timeline
frontend/src/components/user/AnswerCards.tsx - WebSocket API
backend/app/api/ws.py - Alexa endpoint
backend/app/api/routes_alexa.py
Session and planning
- Session manager
backend/app/session/manager.py - Clause extraction
backend/app/nlu/clause_extractor.py - Revisioned plans
backend/app/planning/query_plan.py - Plan invalidation
backend/app/planning/invalidation.py - Task pipeline
backend/app/session/pipeline.py - Retrieval orchestrator
backend/app/planning/orchestrator.py
Retrieval and answers
- Retrieval sources
backend/app/retrieval - Manual search
backend/app/retrieval/manual_search.py - Manual ingestion
backend/app/retrieval/manual_ingest.py - Evidence and precedence
backend/app/evidence - Answer composer
backend/app/answer/composer.py - Optional phrasing
backend/app/llm
Device integrations
- Live state store
backend/app/state/live_store.py - Home simulator
backend/app/devices/simulator.py - SmartThings integration
backend/app/devices/smartthings - Command gate
backend/app/devices/commands.py
Other components
- User
- Alexa skill
- SmartThings
- Manual files
- Gemini API
Connections 34
- User → Assistant UI: asks or corrects
- Assistant UI → Speech input: captures speech
- Speech input → WebSocket client: streams transcripts
- Assistant UI → WebSocket client: sends text
- WebSocket client → WebSocket API: sends events
- WebSocket API → Session manager: dispatches input
- Session manager → Clause extraction: extracts clauses
- Session manager → Revisioned plans: updates plan
- Session manager → Task pipeline: applies plan diff
- Task pipeline → Retrieval orchestrator: runs tasks
- Retrieval orchestrator → Retrieval sources: dispatches retrieval
- Retrieval sources → Live state store: reads state
- Retrieval sources → Manual search: searches manuals
- Manual ingestion → Manual search: indexes sections
- Manual files → Manual ingestion: provides content
- Retrieval orchestrator → Evidence and precedence: stores results
- Task pipeline → Answer composer: composes cards
- Answer composer → Evidence and precedence: resolves facts
- Answer composer → Optional phrasing: requests phrasing
- Optional phrasing → Gemini API: phrases facts
- Task pipeline → WebSocket API: publishes updates
- WebSocket API → WebSocket client: streams updates
- WebSocket client → Cards and timeline: updates cards
- Live state store → Session manager: notifies device changes
- Session manager → Plan invalidation: checks material changes
- Plan invalidation → Revisioned plans: invalidates tasks
- Home simulator → Live state store: emits state changes
- SmartThings → SmartThings integration: sends signed events
- SmartThings integration → Live state store: applies device events
- Session manager → Command gate: executes confirmed command
- Command gate → Home simulator: controls simulated device
- Command gate → SmartThings integration: controls real device
- Alexa skill → Alexa endpoint: sends utterance
- Alexa endpoint → Session manager: answers request