How rangga2077/pro_trial works
CoCI is a local, gesture-driven cooking assistant: camera frames are analyzed into actions, sent to an Electron/React interface, and used to navigate recipes and cooking steps. The documented recipe-management, camera-control, projection, and optional culinary-LLM capabilities are represented as separate branches. The graph follows the README and sampled source; several endpoint, storage, and UI implementation details were not sampled, so uncertain connections are omitted.
Main entry points: pipeline.py CV pipeline · main.py FastAPI app · WebSocketProvider.tsx WebSocket provider · RecipeManager.tsx Recipe manager
Components 27
Backend runtime
- FastAPI app
backend/app/main.py - General API
backend/app/api/v1/endpoints/endpoints.py - Camera API
backend/app/api/v1/endpoints/camera.py - WebSocket route
backend/app/api/v1/endpoints/stream.py - Realtime service
backend/app/services/realtime.py
Vision and actions
- CV pipeline
backend/app/engines/cv/pipeline.py - Object detection
backend/app/engines/cv/yolo.py - Gesture recognition
backend/app/engines/cv/gesture.py - Gesture interpreter
backend/app/services/gesture_actions.py
Recipes and assistant
- Recipe API
backend/app/api/v1/endpoints/recipes.py - Recipe loader
backend/app/services/recipe_loader.py - Recipe files
- Conversation handler
backend/app/engines/llm/handlers.py - LLM router
backend/app/engines/llm/router.py - LLM client
backend/app/engines/llm/client.py
Cooking interface
- Electron shell
frontend/electron/main.ts - React app
frontend/src/App.tsx - WebSocket provider
frontend/src/context/WebSocketProvider.tsx - Recipe manager
frontend/src/components/RecipeManager.tsx - Step display
frontend/src/components/RecipeStep.tsx - CV overlay
frontend/src/components/OverlayCanvas.tsx - Display settings
frontend/src/components/DisplaySettings.tsx
Other components
- Cook
- Ollama
- External LLM
- Display hardware
- Camera
Connections 29
- FastAPI app → General API: mounts routes
- FastAPI app → Camera API: mounts routes
- FastAPI app → Recipe API: mounts routes
- FastAPI app → WebSocket route: mounts route
- FastAPI app → CV pipeline: starts pipeline
- Camera → CV pipeline: provides frames
- CV pipeline → Object detection: tracks objects
- CV pipeline → Gesture recognition: recognizes gestures
- CV pipeline → Gesture interpreter: interprets gestures
- CV pipeline → Realtime service: broadcasts updates
- Realtime service → WebSocket provider: streams updates
- WebSocket provider → WebSocket route: connects by WebSocket
- React app → WebSocket provider: provides context
- React app → Recipe manager: renders
- React app → CV overlay: renders
- React app → Display settings: renders
- Cook → Recipe manager: uses gestures
- Recipe manager → Recipe API: requests recipes
- Recipe API → Recipe loader: loads and saves
- Recipe loader → Recipe files: reads and writes
- Recipe manager → Step display: shows current step
- WebSocket provider → Recipe manager: provides updates
- WebSocket provider → CV overlay: provides updates
- Display settings → Electron shell: requests display control
- Electron shell → Display hardware: moves window
- Conversation handler → LLM client: generates response
- LLM router → LLM client: routes simple queries
- LLM client → Ollama: sends generation request
- LLM router → External LLM: simulates substitution route