How adaptivemotorcontrollab/amadeusgpt works
Project type and purpose
• This is an open-source Python library and application for interactive animal behavior analysis via natural-language–to–code translation, powered by large language models (LLMs) and DeepLabCut/SuperAnimal.
• It provides both a Streamlit web UI and a programmatic API (importable Python package) for users to load videos/keypoint files, pose queries in natural language, and receive executable analysis code and visualizations.
File structure
• Top-level scripts and configs:
• launch_amadeusGPT.py / Makefile / Streamlit config → entrypoints
• pyproject.toml, setup.cfg → packaging
• conda/ → environment definitions
• Core package (amadeusgpt/):
• app.py, main.py → orchestration (CLI & GUI bootstrapping)
• project.py, config.py, app_utils.py → project creation & configuration
• managers/ → Manager classes (animal, event, object, relationship, GUI, visual)
• behavior_analysis/ → analysis factory and algorithms
• analysis_objects/ → domain models (Animal, Event, Relationship, Visualization, etc.)
• integration_modules/ → plugins for embedding, kinematics, dimensionality reduction
• programs/ → registries for task programs and API integrations
• system_prompts/ → LLM prompting templates for code generation, debugging, explanations
• utils.py, logger.py → support utilities
• version.py
• Supporting directories:
• static/ → CSS, video & image assets for web UI
• notebooks/ → Jupyter demos
• examples/ → sample data (video + .h5 + config)
• tests/ → pytest test suite
• CI/CD:
• .github/workflows → linting, testing, deployment
• Makefile → make app, make test, etc.
Extract architecture insights from the README
• Users supply an OpenAI API key and install via conda or pip.
• Two main modes: Jupyter notebooks or Streamlit web app.
• High-level flow: create_project() → instantiate AMADEUS → amadeus.step(query) → LLM generates Python code → code execution sandbox → results parsed & visualized.
1. User Interfaces
– CLI / Python API
– Streamlit Web App
– Jupyter Notebooks
2. Core Engine
– Project & Config Manager
– AMADEUS Orchestrator
– Manager Layer (AnimalManager, EventManager, RelationshipManager, VisualManager, GUIManager)
– BehaviorAnalysis Factory & Algorithms
– Integration Module Hub (plugin loader for embedding, kinematics)
– Program Registry (task programs, API registry, sandbox)
– Analysis Object Models
– System Prompt Templates
– Utilities & Logger
3. External Services & Data
– OpenAI API (LLM)
– Video & Keypoint Files (DeepLabCut / SuperAnimal)
– Results Folder (code outputs, figures, JSON)
4. Deployment & CI
– Conda Environment
– GitHub Actions (codespell, pytest, test_and_deploy)
b. Relationships and interactions
• The Orchestrator invokes:
• Project Manager to set up data folders & config
• Manager Layer to load data, instantiate analysis objects
• BehaviorAnalysis Factory to choose appropriate analysis based on user query
• IntegrationModuleHub to load plugins
• SystemPrompts to craft LLM requests
• LLM Connector to send/receive OpenAI API calls
• Program Registry / Sandbox to execute generated code
• GUIManager / VisualManager to render plots & tables
• Data flows: Video/keypoints → Manager layer → analysis objects → LLM → generated code → sandboxed execution → parse_result → visualization → user.
c. Architectural patterns & design principles
• Layered architecture (UI → Orchestrator → Managers → Analysis → Data).
• Plugin/extension pattern for integration modules (embedding, kinematics).
• Factory pattern (analysis_factory) to select analysis strategy.
• Registry pattern for program tasks and API integrations.
• Separation of concerns: prompting logic separated in system_prompts.
• Test-driven design (comprehensive tests under tests/).
d. Key technologies & frameworks
• Python 3.10+ with libraries: Streamlit, PyTorch/DeepLabCut, SuperAnimal, PyTables, YAML.
• OpenAI API for LLM calls.
• Conda for environment management.
• GitHub Actions for CI.
• Jupyter for demos.