How vincenzo-afk/iris works
IRIS is a local desktop assistant: users select a mode, then receive screen descriptions, contextual chat, hover explanations, bounded computer-use tasks, or voice-driven responses and actions. The graph preserves those distinct modes and the agent’s memory, web, task, and desktop-action capabilities. Several links are documented in the README or visible in indexed imports; unsampled implementations are represented only at their supported directory or module paths.
Main entry points: agent.py Task agent · main.py Mode selector · vision.py Screen vision · voice.py Voice listener
Components 21
Interaction modes
- Mode selector
main.py - Screen watch
core/vision.py - Contextual chat
core/agent.py - Teach mode
overlay/teach_mode.py - Voice listener
core/voice.py
Screen understanding
- Screen vision
core/vision.py - Screen tools
tools/screen_tools.py
Agent and memory
- Task agent
core/agent.py - Assistant memory
core/memory.py - Task list
tools/task_tools.py - Web research
tools/web_tools.py
Desktop capabilities
- Input automation
core/automation.py - Input tools
tools/input_tools.py - System tools
tools/system_tools.py - Cursor overlay
overlay/cursor_widget.py
Runtime configuration
- Runtime settings
config.py
Other components
- User
- Google Gemini
- Mem0 storage
- Desktop session
- Audio devices
Connections 29
- User → Mode selector: selects mode
README.md - Mode selector → Screen watch: starts watch
main.py - Mode selector → Contextual chat: routes chat
main.py - Mode selector → Teach mode: starts teaching
main.py - Mode selector → Task agent: runs task
main.py - Mode selector → Voice listener: starts listening
main.py - Screen watch → Screen vision: analyzes frames
README.md - Teach mode → Screen vision: analyzes hover
overlay/teach_mode.py - Teach mode → Cursor overlay: shows explanation
overlay/teach_mode.py - Voice listener → Mode selector: delivers commands
main.py - Voice listener → Audio devices: records and speaks
README.md - Contextual chat → Google Gemini: answers prompts
README.md - Screen vision → Google Gemini: describes screen
README.md - Task agent → Google Gemini: requests tool calls
README.md - Task agent → Assistant memory: loads context
core/agent.py - Assistant memory → Mem0 storage: stores memories
README.md - Task agent → Screen vision: observes and verifies
core/agent.py - Task agent → Input automation: dispatches actions
core/agent.py - Task agent → Task list: reads task list
core/agent.py - Task agent → System tools: opens applications
core/agent.py - Task agent → Web research: uses web tools
core/agent.py - Input automation → Desktop session: controls input
README.md - System tools → Desktop session: operates system
tools/system_tools.py - Input tools → Input automation: delegates input
tools/input_tools.py - Screen tools → Screen vision: uses capture
tools/screen_tools.py - Mode selector → Runtime settings: uses settings
main.py - Task agent → Runtime settings: uses settings
core/agent.py - Screen vision → Runtime settings: uses settings
core/vision.py - Voice listener → Runtime settings: uses settings
core/voice.py