How autogluon/autogluon-assistant works
Project Type & Purpose
• This is a full-stack, multi-agent system (AutoGluon Assistant / MLZero) for end-to-end machine learning automation.
• It provides both a CLI (“mlzero” entry point) and a Streamlit-based Web UI, orchestrating specialized “agents” that generate and execute code via large language models (LLMs) to build ML pipelines with zero human intervention.
Top-Level Components
A. Presentation Layer
• CLI (run.py + src/autogluon/assistant/cli/app.py)
• Web UI (src/autogluon/assistant/webui/, Streamlit front-end + lightweight FastAPI backend)
B. Core Orchestrator
• Manager (src/autogluon/assistant/managers/manager.py) – dispatches tasks to agents, tracks progress, coordinates memory modules
C. Agents
• TaskDescriptorAgent
• RetrieverAgent
• DataPerceptionAgent
• ToolSelectorAgent
• CoderAgent / ExecuterAgent
• ErrorAnalyzerAgent
• RerankerAgent
(each under src/autogluon/assistant/agents/)
D. LLM Layer
• BaseChat + factory (llm_factory.py)
• Implementations: OpenAI, AzureOpenAI, Anthropic, AWS Bedrock (src/autogluon/assistant/llm/)
E. Tooling & Knowledge Base
• Tools Registry (src/autogluon/assistant/tools_registry/)
– catalogs, JSON descriptors, tutorials & “condensed_tutorials” for each domain (tabular, multimodal, embedding, timeseries)
• Utility modules (indexing.py, registry.py, utils.py)
F. Execution & Memory
• Dual-memory modules (in Manager / agents’ memory handling)
• Code execution environment (shell scripts invoked by ExecuterAgent)
• LogProcessor (captures stdout/stderr, feeds back to ErrorAnalyzerAgent)
G. Configuration & Logging
• YAML config (default.yaml) & constants
• rich_logging for structured logs
H. External Services & Data Stores
• LLM APIs (AWS Bedrock, OpenAI, Anthropic, Azure)
• Local file system (input dataset folder, generated code, model artifacts)
I. CI & Testing
• GitHub Actions workflows (.github/workflows/*.yml)
• Unit & integration tests (tests/unittests/)
• Release automation (pypi_release.yml, nightly_release.yml)
Relationships & Data Flows
• User invokes CLI or opens Web UI → Presentation Layer
• Presentation Layer calls Manager → Manager spins up Agents in sequence
• Agents invoke LLM Layer via LLMFactory → receive prompt completions
• RetrieverAgent & ToolSelectorAgent query Tools Registry for domain-specific snippets & docs
• CoderAgent generates code → ExecuterAgent runs it in the OS shell → LogProcessor collects execution logs
• ErrorAnalyzerAgent examines logs → feeds corrections back into CoderAgent
• RerankerAgent orders candidate solutions → Manager stores to Output folder
• Throughout, Manager uses Memory modules to track context & past interactions
• Final ML solution artifacts (trained models, evaluation reports) are saved locally
Architectural Patterns & Principles
• Multi-Agent Orchestration (each agent has single responsibility)
• Factory Pattern for LLM instantiation (llm_factory.py)
• Plugin/Registry Pattern for tools & tutorials
• Separation of Concerns: Presentation vs. Orchestration vs. Execution vs. Storage
• Feedback Loop: code generation ↔ execution feedback ↔ error analysis