How agenttankos/tankwork works
Project Type & Purpose
• TankWork is a desktop agent framework aimed at developers and researchers experimenting with autonomous computer control. It offers two distinct modes – “Computer Use Mode” for direct system control and “Assistant Mode” for screen analysis with skills like ticker analysis.
• The project integrates components for computer vision, voice processing, command handling, and agent configuration, all orchestrated by a central entry point (main.py).
File Structure
• Core Functionality (core directory):
• Avatar Subsystem: Contains avatar event handling, manager, and models used for configuring and working with agent visual/voice representations.
• Command Handling: The command accelerator, command manager, and dedicated modules (e.g., core/command_accelerator, core/command_manager) indicate a subsystem for interpreting and routing commands.
• Computer Use & Interface: Modules such as computer_use_factory, computer_use_interface, and computer_use_providers (e.g., the tank implementation in computer_use_providers) suggest pluggable implementations for direct computer control.
• Narrative Processing: The narrative_processor.py module sets up real-time text feedback and analysis.
• Example Skill: The ticker analysis under core/skills demonstrates one of possibly many skills that process screen content.
• Voice Integration: Modules for voice and voice_commands integrate voice recognition and natural language processing.
• Screenshot Functionality: Provides screen capture capabilities to enable vision analysis.
• UI Layer (ui directory):
• Contains user interface components like app.py, loading_eyes.py, and notification.py. This layer likely handles feedback, visualization, and user interaction.
• Configuration & Assets:
• The config directory holds key configuration files for avatars, general settings, and logging.
• Assets (including avatars, images, and videos) provide the necessary multimedia components for the visual identity of each agent.
Main Components & Their Interactions
• Main Entry Point (main.py): Acts as the bootstrapper that initializes configurations, core functionalities, and the UI.
• Core Modules:
• Avatar Management: Handles agent configuration and interaction with customizable avatars.
• Command Processing: Receives and interprets both voice and text commands. It further routes these commands to the appropriate submodules (e.g., computer use providers or skill processors).
• Computer Use Providers: Implements the logic for direct computer control (using various providers, with “tank” as one default).
• Narrative & Feedback: Processes screen captures and logs audio-visual feedback, ready to be presented by the UI.
• Voice System: Integrates with ElevenLabs and other APIs to enable natural language voice command services.
• UI Components: Visual interface modules that present information (action logs, notifications, visual cues like “loading eyes”) and accept user input.
Architectural Patterns & Principles
• Modular Design: The project’s structure is clearly divided into independent modules (e.g., core, ui, config, assets). Highlighting this separation ensures clarity regarding which modules manage particular responsibilities.
• Plugin/Provider Pattern: The computer use providers (and, by extension, custom skills) suggest a pluggable system where different “implementation” modules can be swapped or extended.
• Configurable Agents: The extensive agent configuration system indicates a design that values extensibility and customizability.
• Separation of Concerns: Interaction (voice and textual) and computational tasks (command processing, vision analysis, computer control) are encapsulated in different layers.
Final Checklist for the Principal Software Engineer
• Core: Avatar (with events, manager, models), Command Accelerator & Manager, Computer Use (factory, interface, providers), Narrative Processor, Screenshot, Skills (Ticker Analysis), Voice & Commands.
• UI: App, Loading Eyes, Notification.
• Config: Configuration modules (general config, avatar config, logging config).
• Assets: Avatars (images, videos, etc.).