# dinh-huu-phuc/friday architecture

> Architecture diagram of the GitHub repository dinh-huu-phuc/friday, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/dinh-huu-phuc/friday
- Repository: https://github.com/dinh-huu-phuc/friday

## Overview

FRIDAY is a Windows-focused AI assistant with a local FastAPI interface and capabilities spanning conversational assistance, retrieval, vision, device control, and integrations. The main visible interaction path is a user sending chat over the UI WebSocket to the agent-console service; README also documents hands-free voice and external speech/LLM providers. The graph retains major subsystems named in the README and tree, but limits internal wiring to relationships supported by the sampled source or explicit documentation; unsampled subsystem paths are exact enclosing directories.

Main entry points: [`friday-tony-stark-demo/friday/runtime/decision_engine.py`](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/runtime/decision_engine.py) (Decision Engine), [`friday-tony-stark-demo/friday/app/perception`](https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/perception) (Perception Services), [`friday-tony-stark-demo/friday/src/UI/routes.py`](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/src/UI/routes.py) (UI Chat Routes), [`friday-tony-stark-demo/friday/app/agent_console`](https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/agent_console) (Agent Console)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interaction["User Interaction"]
  node_ui_routes["UI Chat Routes<br/>[routes.py]"]
  node_agent_console["Agent Console"]
  node_speech["Speech Services<br/>[tts_service.py]"]
end

subgraph group_reasoning["Assistant Reasoning"]
  node_decision["Decision Engine<br/>[decision_engine.py]"]
  node_rag["RAG Pipeline<br/>[rag_pipeline.py]"]
  node_language["Language Manager<br/>[manager.py]"]
end

subgraph group_perception["Vision And Spatial"]
  node_camera["Camera Manager<br/>[camera_manager.py]"]
  node_perception["Perception Services"]
  node_relations["Scene Relations<br/>[engine.py]"]
  node_spatial["Gesture Engine<br/>[gesture_engine.py]"]
end

subgraph group_actions["Device Actions"]
  node_computer["Computer Control"]
  node_browser["Browser Automation<br/>[service.py]"]
  node_sleep["Sleep Display<br/>[process_manager.py]"]
  node_calendar["Calendar Reminders"]
  node_research["Live Search"]
  node_code_map["Code Map"]
end

subgraph group_integrations["External Integrations"]
  node_social["Social Connectors"]
  node_ollama["Local Vision Model"]
  node_supabase[("Supabase Storage")]
end

node_user(("User"))
node_browser_actor["Chrome"]
node_stt["Speech Recognition"]
node_llm["Configured LLM"]
node_tts["Configured TTS"]

node_user -->|"sends chat"| node_ui_routes
node_ui_routes -->|"dispatches chat"| node_agent_console
node_agent_console -->|"uses speech"| node_speech
node_user -.->|"provides voice"| node_stt
node_stt -.->|"transcribes input"| node_llm
node_llm -.->|"generates reply"| node_tts
node_decision -->|"loads messages"| node_language
node_decision -->|"retrieves context"| node_rag
node_camera -->|"captures frames"| node_perception
node_perception -->|"infers relations"| node_relations
node_spatial -->|"classifies gestures"| node_perception
node_browser -->|"controls browser"| node_browser_actor
node_decision -.->|"uses vision"| node_ollama
node_supabase -->|"stores objects"| node_supabase

click node_ui_routes "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/src/UI/routes.py"
click node_agent_console "https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/agent_console"
click node_speech "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/agent_console/tts_service.py"
click node_decision "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/runtime/decision_engine.py"
click node_rag "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/core/rag/rag_pipeline.py"
click node_language "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/language/manager.py"
click node_camera "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/perception/camera/camera_manager.py"
click node_perception "https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/perception"
click node_relations "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/perception/relation/engine.py"
click node_spatial "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/spatial/service/gesture_engine.py"
click node_computer "https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/computer"
click node_browser "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/browser_automation/service.py"
click node_sleep "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/sleep_display/process_manager.py"
click node_calendar "https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/calendar"
click node_research "https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/research"
click node_code_map "https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/code_map"
click node_social "https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/discord"
click node_supabase "https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/core/db/services/supabase_storage.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ui_routes,node_agent_console,node_speech,node_user toneBlue
class node_decision,node_rag,node_language toneAmber
class node_camera,node_perception,node_relations,node_spatial toneMint
class node_computer,node_browser,node_sleep,node_calendar,node_research,node_code_map toneRose
class node_social,node_ollama,node_supabase,node_browser_actor,node_stt,node_llm,node_tts toneIndigo
```

## Components

### User Interaction

- [UI Chat Routes](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/src/UI/routes.py): `friday-tony-stark-demo/friday/src/UI/routes.py`
- [Agent Console](https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/agent_console): `friday-tony-stark-demo/friday/app/agent_console`
- [Speech Services](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/agent_console/tts_service.py): `friday-tony-stark-demo/friday/app/agent_console/tts_service.py`

### Assistant Reasoning

- [Decision Engine](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/runtime/decision_engine.py): `friday-tony-stark-demo/friday/runtime/decision_engine.py`
- [RAG Pipeline](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/core/rag/rag_pipeline.py): `friday-tony-stark-demo/friday/core/rag/rag_pipeline.py`
- [Language Manager](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/language/manager.py): `friday-tony-stark-demo/friday/language/manager.py`

### Vision And Spatial

- [Camera Manager](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/perception/camera/camera_manager.py): `friday-tony-stark-demo/friday/app/perception/camera/camera_manager.py`
- [Perception Services](https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/perception): `friday-tony-stark-demo/friday/app/perception`
- [Scene Relations](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/perception/relation/engine.py): `friday-tony-stark-demo/friday/app/perception/relation/engine.py`
- [Gesture Engine](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/spatial/service/gesture_engine.py): `friday-tony-stark-demo/friday/app/spatial/service/gesture_engine.py`

### Device Actions

- [Computer Control](https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/computer): `friday-tony-stark-demo/friday/app/computer`
- [Browser Automation](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/browser_automation/service.py): `friday-tony-stark-demo/friday/app/browser_automation/service.py`
- [Sleep Display](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/app/sleep_display/process_manager.py): `friday-tony-stark-demo/friday/app/sleep_display/process_manager.py`
- [Calendar Reminders](https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/calendar): `friday-tony-stark-demo/friday/app/calendar`
- [Live Search](https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/research): `friday-tony-stark-demo/friday/app/research`
- [Code Map](https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/code_map): `friday-tony-stark-demo/friday/app/code_map`

### External Integrations

- [Social Connectors](https://github.com/dinh-huu-phuc/friday/tree/master/friday-tony-stark-demo/friday/app/discord): `friday-tony-stark-demo/friday/app/discord`
- **Local Vision Model**
- [Supabase Storage](https://github.com/dinh-huu-phuc/friday/blob/master/friday-tony-stark-demo/friday/core/db/services/supabase_storage.py): `friday-tony-stark-demo/friday/core/db/services/supabase_storage.py`

### Other components

- **User**
- **Chrome**
- **Speech Recognition**
- **Configured LLM**
- **Configured TTS**

## Connections

- User → UI Chat Routes: sends chat
- UI Chat Routes → Agent Console: dispatches chat
- Agent Console → Speech Services: uses speech
- User → Speech Recognition: provides voice
- Speech Recognition → Configured LLM: transcribes input
- Configured LLM → Configured TTS: generates reply
- Decision Engine → Language Manager: loads messages
- Decision Engine → RAG Pipeline: retrieves context
- Camera Manager → Perception Services: captures frames
- Perception Services → Scene Relations: infers relations
- Gesture Engine → Perception Services: classifies gestures
- Browser Automation → Chrome: controls browser
- Decision Engine → Local Vision Model: uses vision
- Supabase Storage → Supabase Storage: stores objects