# adaptivemotorcontrollab/amadeusgpt architecture

> Architecture diagram of the GitHub repository adaptivemotorcontrollab/amadeusgpt, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/adaptivemotorcontrollab/amadeusgpt
- Repository: https://github.com/adaptivemotorcontrollab/amadeusgpt

## Overview

##### Project type and purpose
- This is an open-source Python library and application for interactive animal behavior analysis via natural-language–to–code translation, powered by large language models (LLMs) and DeepLabCut/SuperAnimal.
- It provides both a Streamlit web UI and a programmatic API (importable Python package) for users to load videos/keypoint files, pose queries in natural language, and receive executable analysis code and visualizations.

##### File structure
- Top-level scripts and configs:
- launch_amadeusGPT.py / Makefile / Streamlit config → entrypoints
- pyproject.toml, setup.cfg → packaging
- conda/ → environment definitions
- Core package (`amadeusgpt/`):
- app.py, main.py → orchestration (CLI & GUI bootstrapping)
- project.py, config.py, app_utils.py → project creation & configuration
- managers/ → Manager classes (animal, event, object, relationship, GUI, visual)
- behavior_analysis/ → analysis factory and algorithms
- analysis_objects/ → domain models (Animal, Event, Relationship, Visualization, etc.)
- integration_modules/ → plugins for embedding, kinematics, dimensionality reduction
- programs/ → registries for task programs and API integrations
- system_prompts/ → LLM prompting templates for code generation, debugging, explanations
- utils.py, logger.py → support utilities
- version.py
- Supporting directories:
- static/ → CSS, video & image assets for web UI
- notebooks/ → Jupyter demos
- examples/ → sample data (video + .h5 + config)
- tests/ → pytest test suite
- CI/CD:
- .github/workflows → linting, testing, deployment
- Makefile → `make app`, `make test`, etc.

##### Extract architecture insights from the README
- Users supply an OpenAI API key and install via conda or pip.
- Two main modes: Jupyter notebooks or Streamlit web app.
- High-level flow: `create_project()` → instantiate `AMADEUS` → `amadeus.step(query)` → LLM generates Python code → code execution sandbox → results parsed & visualized.

1. User Interfaces
– CLI / Python API
– Streamlit Web App
– Jupyter Notebooks
2. Core Engine
– Project & Config Manager
– AMADEUS Orchestrator
– Manager Layer (AnimalManager, EventManager, RelationshipManager, VisualManager, GUIManager)
– BehaviorAnalysis Factory & Algorithms
– Integration Module Hub (plugin loader for embedding, kinematics)
– Program Registry (task programs, API registry, sandbox)
– Analysis Object Models
– System Prompt Templates
– Utilities & Logger
3. External Services & Data
– OpenAI API (LLM)
– Video & Keypoint Files (DeepLabCut / SuperAnimal)
– Results Folder (code outputs, figures, JSON)
4. Deployment & CI
– Conda Environment
– GitHub Actions (codespell, pytest, test_and_deploy)

b. Relationships and interactions
- The Orchestrator invokes:
- Project Manager to set up data folders & config
- Manager Layer to load data, instantiate analysis objects
- BehaviorAnalysis Factory to choose appropriate analysis based on user query
- IntegrationModuleHub to load plugins
- SystemPrompts to craft LLM requests
- LLM Connector to send/receive OpenAI API calls
- Program Registry / Sandbox to execute generated code
- GUIManager / VisualManager to render plots & tables
- Data flows: Video/keypoints → Manager layer → analysis objects → LLM → generated code → sandboxed execution → parse_result → visualization → user.

c. Architectural patterns & design principles
- Layered architecture (UI → Orchestrator → Managers → Analysis → Data).
- Plugin/extension pattern for integration modules (embedding, kinematics).
- Factory pattern (`analysis_factory`) to select analysis strategy.
- Registry pattern for program tasks and API integrations.
- Separation of concerns: prompting logic separated in `system_prompts`.
- Test-driven design (comprehensive tests under `tests/`).

d. Key technologies & frameworks
- Python 3.10+ with libraries: Streamlit, PyTorch/DeepLabCut, SuperAnimal, PyTables, YAML.
- OpenAI API for LLM calls.
- Conda for environment management.
- GitHub Actions for CI.
- Jupyter for demos.

## Diagram (Mermaid)

```mermaid
flowchart TB
  subgraph "User Interfaces"
    direction TB
    CLI["CLI / Python API"]:::ui
    Streamlit["Streamlit Web App"]:::ui
    Notebook["Jupyter Notebooks"]:::ui
  end

  subgraph "Core Engine"
    direction TB
    subgraph "Orchestration"
      direction TB
      Launcher["launch_amadeusGPT.py"]:::core
      AppPy["app.py"]:::core
      MainPy["main.py"]:::core
    end
    subgraph "Project & Config"
      direction TB
      ProjectMgr["project.py"]:::core
      Config["config.py"]:::core
      AppUtils["app_utils.py"]:::core
    end
    subgraph "Managers"
      direction TB
      BaseMgr["base.py"]:::core
      AnimalMgr["animal_manager.py"]:::core
      EventMgr["event_manager.py"]:::core
      ObjectMgr["object_manager.py"]:::core
      RelMgr["relationship_manager.py"]:::core
      GUIMgr["gui_manager.py"]:::core
      VisMgr["visual_manager.py"]:::core
    end
    subgraph "BehaviorAnalysis"
      direction TB
      Factory["analysis_factory.py"]:::core
      AnimalAnalysis["animal_behavior_analysis.py"]:::core
      Identifier["identifier.py"]:::core
    end
    subgraph "Analysis Models"
      direction TB
      BaseObj["base.py"]:::core
      AnimalObj["animal.py"]:::core
      EventObj["event.py"]:::core
      ObjectObj["object.py"]:::core
      RelObj["relationship.py"]:::core
      VisObj["visualization.py"]:::core
      LLMObj["llm.py"]:::core
      ModelObj["model.py"]:::core
    end
    subgraph "Integration Modules"
      direction TB
      Hub["integration_module_hub.py"]:::core
      Cebra["cebra.py"]:::core
      Transform["transform.py"]:::core
      UMAP["umap.py"]:::core
      Gait["gait.py"]:::core
    end
    subgraph "Programs & Sandbox"
      direction TB
      TaskRegistry["task_program_registry.py"]:::core
      APIRegistry["api_registry.py"]:::core
      Sandbox["sandbox.py"]:::core
    end
    subgraph "System Prompts"
      direction TB
      CodeGen["code_generator.py"]:::core
      Diagnosis["diagnosis.py"]:::core
      Explainer["explainer.py"]:::core
      Debug["self_debug.py"]:::core
      VisLLM["visual_llm.py"]:::core
    end
    subgraph "Utils & Logging"
      direction TB
      Utils["utils.py"]:::core
      Logger["logger.py"]:::core
    end
    Version["version.py"]:::core
  end

  subgraph "External & Data"
    direction TB
    OpenAI["OpenAI API"]:::external
    VideoData["Video & Keypoint Files"]:::external
    Results["Results Folder"]:::external
  end

  subgraph "Deployment & CI"
    direction TB
    CondaEnv["Conda Environment"]:::dev
    Codespell["codespell.yml"]:::dev
    PytestCI["pytest.yml"]:::dev
    DeployCI["test_and_deploy.yaml"]:::dev
  end

  CLI --> MainPy
  Streamlit --> AppPy
  Notebook --> AppPy

  AppPy --> ProjectMgr
  MainPy --> ProjectMgr

  ProjectMgr --> BaseMgr
  BaseMgr --> AnimalMgr
  BaseMgr --> EventMgr
  BaseMgr --> ObjectMgr
  BaseMgr --> RelMgr

  AnimalMgr --> Factory
  EventMgr --> Factory
  ObjectMgr --> Factory

  Factory --> CodeGen
  CodeGen --> OpenAI
  OpenAI --> Sandbox
  Sandbox --> Results
  Results --> VisMgr
  VisMgr --> Streamlit

  Factory --> Hub
  Hub --> Cebra
  Hub --> Transform
  Hub --> UMAP
  Hub --> Gait

  subgraph Packaging
    PYProj["pyproject.toml"]:::dev
    SetupCfg["setup.cfg"]:::dev
  end

  subgraph Environments
    CondaYML["amadeusGPT.yml"]:::dev
    InstallCPU["install_cpu.sh"]:::dev
    InstallGPU["install_gpu.sh"]:::dev
    InstallMin["install_minimal.sh"]:::dev
  end

  CondaEnv --> CondaYML
  CondaEnv --> InstallCPU
  CondaEnv --> InstallGPU
  CondaEnv --> InstallMin

  classDef ui fill:#ADD8E6,stroke:#333,stroke-width:1px
  classDef core fill:#90EE90,stroke:#333,stroke-width:1px
  classDef external fill:#FFD580,stroke:#333,stroke-width:1px
  classDef dev fill:#D3D3D3,stroke:#333,stroke-width:1px

  click Launcher "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/launch_amadeusGPT.py"
  click AppPy "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/app.py"
  click MainPy "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/main.py"
  click ProjectMgr "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/project.py"
  click Config "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/config.py"
  click AppUtils "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/app_utils.py"
  click BaseMgr "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/managers/base.py"
  click AnimalMgr "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/managers/animal_manager.py"
  click EventMgr "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/managers/event_manager.py"
  click ObjectMgr "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/managers/object_manager.py"
  click RelMgr "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/managers/relationship_manager.py"
  click GUIMgr "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/managers/gui_manager.py"
  click VisMgr "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/managers/visual_manager.py"
  click Factory "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/behavior_analysis/analysis_factory.py"
  click AnimalAnalysis "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/behavior_analysis/animal_behavior_analysis.py"
  click Identifier "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/behavior_analysis/identifier.py"
  click BaseObj "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/analysis_objects/base.py"
  click AnimalObj "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/analysis_objects/animal.py"
  click EventObj "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/analysis_objects/event.py"
  click ObjectObj "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/analysis_objects/object.py"
  click RelObj "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/analysis_objects/relationship.py"
  click VisObj "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/analysis_objects/visualization.py"
  click LLMObj "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/analysis_objects/llm.py"
  click ModelObj "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/analysis_objects/model.py"
  click Hub "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/integration_module_hub.py"
  click Cebra "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/integration_modules/embedding/cebra.py"
  click Transform "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/integration_modules/embedding/transform.py"
  click UMAP "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/integration_modules/embedding/umap.py"
  click Gait "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/integration_modules/kinematics/gait.py"
  click TaskRegistry "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/programs/task_program_registry.py"
  click APIRegistry "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/programs/api_registry.py"
  click Sandbox "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/programs/sandbox.py"
  click CodeGen "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/system_prompts/code_generator.py"
  click Diagnosis "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/system_prompts/diagnosis.py"
  click Explainer "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/system_prompts/explainer.py"
  click Debug "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/system_prompts/self_debug.py"
  click VisLLM "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/system_prompts/visual_llm.py"
  click Utils "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/utils.py"
  click Logger "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/logger.py"
  click Version "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/amadeusgpt/version.py"
  click PYProj "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/pyproject.toml"
  click SetupCfg "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/setup.cfg"
  click CondaYML "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/conda/amadeusGPT.yml"
  click InstallCPU "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/conda/install_cpu.sh"
  click InstallGPU "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/conda/install_gpu.sh"
  click InstallMin "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/conda/install_minimal.sh"
  click Codespell "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/.github/workflows/codespell.yml"
  click PytestCI "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/.github/workflows/pytest.yml"
  click DeployCI "https://github.com/adaptivemotorcontrollab/amadeusgpt/blob/main/.github/workflows/test_and_deploy.yaml"
```
