How hazyresearch/minions works
Main Components
• Frontend/Demo Interface: This includes the “app.py” that launches a Streamlit demo, the “minions_cli.py” for CLI interaction, and “minions.ipynb” for notebook usage. These interfaces allow a user to trigger the protocol.
• Core Protocol Module: The core modules (“minion.py”, “minions.py”, “minions_mcp.py”) implement the logic for orchestrating the back-and-forth communication between local and remote models.
• Clients Library: The “minions/clients/” directory encapsulates integrations with many cloud services and on-device model servers (e.g., “ollama”, “openai”, “azure_openai”, “together”, “tokasaurus”, etc.). Each client acts as an abstraction for handling API interactions.
• Utilities and Helpers: The “minions/utils/” directory – including helpers for document processing, energy tracking, retrieval, etc. – which provide ancillary functionalities.
• Example Code & Prompts: Directories “minions/examples/” and “minions/prompts/” demonstrate how to set up tasks and run cases, providing real-world scenarios that show the protocol in action.
Relationships and Data Flow
• User inputs a request (via a CLI, GUI, or Notebook).
• The front-end interface directs the input to the core protocol layer.
• The protocol layer calls the appropriate clients (one local and one remote) to process a “context” and “task” in a conversational or round-based manner.
• The clients manage their own API interactions – local requests (via servers like “ollama” or “tokasaurus”) and remote requests (to providers like “OpenAI”, “Azure OpenAI”, etc.), then exchange feedback based on the protocol.
• Utility modules may handle additional processing (like document extraction or energy tracking) throughout this communication cycle.
• Results are then communicated back to the user through the front-end.
Architectural Patterns and Design Principles
• Modular Design: The project is organized into discrete modules (clients, core protocol, utilities) allowing independent updates and potential extensions.
• Plugin-like Integration: Each client is isolated in its own module, following a standardized interface, which resembles a plugin system.
• Extensibility and Abstraction: The protocol classes encapsulate the round-based communication logic, abstracting away the details of different LLM interactions.
• Separation of Concerns: Front-end presentation (CLI/streamlit/notebook), core orchestration, and external service integrations are kept distinct.
Technology Stack and Relevant Details
• Python 3.10-3.11 support (with compatibility notes).
• Integration with external LLM services (OpenAI, Azure OpenAI, Together, etc.) is critical.
• Optional components for specialized hardware support (e.g., using “ollama” on non-NVIDIA setups or “tokasaurus” with NVIDIA GPUs).