# moreagentsisallyouneed/agentforest architecture

> Architecture diagram of the GitHub repository moreagentsisallyouneed/agentforest, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-17). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/moreagentsisallyouneed/agentforest
- Repository: https://github.com/moreagentsisallyouneed/agentforest

## Overview

##### Project type and purpose
- Type: A research-oriented CLI framework for running multi-agent experiments with Large Language Models (LLMs).
- Purpose: Orchestrate N LLM agents to solve tasks (MATH, GSM, MMLU, Chess, HumanEval), collect and evaluate their responses, and compare performance across models (OpenAI GPT on Azure or self-hosted LLM).

##### File structure
- Top level
– README.md: Project overview and setup
– dataset/: Raw problem sets (Chess, GSM, Math, MMLU)
– img/: Illustrative algorithm PDF
– script/: Shell wrappers (run.sh, run_generation_task.sh, run_reasoning_task.sh)
– src/: Core Python modules
- src/ layout
– main.py: Entry point—parses CLI args, dispatches tasks
– dataloader.py: Loads JSON/CSV into in-memory examples
– {chess, gsm, math, mmlu}_task.py: Task-specific prompt construction and parsing
– agent.py & more_agent.py: Abstractions for single-agent vs. multi-agent orchestration
– prompt_lib.py: Reusable prompt templates and system messages
– evaluation.py & utils.py: Scoring and helper routines
– categories.py: Defines question categories or labels
- script/
– run.sh: Wrapper that accepts {AGENT_NUM} {MODEL} {QTYPE} and invokes the correct sub-script
– run_generation_task.sh vs. run_reasoning_task.sh: Shell variants based on question type

##### README for additional insights
- External dependencies:
– human-eval framework (for HumanEval tasks)
– Environment variables for API endpoints and keys (OPENAI_KEY, OPENAI_IP, LLM_IP)
- Modes of operation: Multi-agent (“More Agents”) vs. single-agent runs
- Supported models: Azure-hosted GPT series or open-source LLM via HTTP

a. Main components
1. CLI / Shell Scripts (script/run.sh → run_generation_task.sh or run_reasoning_task.sh)
2. Orchestrator (src/main.py)
3. Data Layer (src/dataloader.py + dataset/*)
4. Task Modules (src/*_task.py)
5. Agent Layer (src/agent.py, src/more_agent.py)
6. Prompt Library (src/prompt_lib.py)
7. LLM Interfaces (src/utils.py handling API calls to Azure or self-hosted LLM)
8. Evaluation Engine (src/evaluation.py + human-eval integration)
9. Output Store (file system or JSON/CSV results)

b. Relationships and interactions
– Shell Scripts invoke the Orchestrator with parameters
– Orchestrator calls Data Layer to load problems
– For each example, Orchestrator uses Task Modules + Prompt Library to build prompts
– Agent Layer spins up N agents; each agent calls the LLM Interface in parallel or sequence
– LLM Interface issues HTTP requests to external LLM services (Azure OpenAI or LLM_IP)
– Responses routed back to Orchestrator, then passed to Evaluation Engine
– Evaluation Engine uses metrics or human-eval scripts, writes out results

c. Architectural patterns
– Modular “Task” plugins: each question type lives in its own module, enabling easy extension
– Agent abstraction: single vs. multi-agent interchangeable via strategy pattern
– Clear separation of concerns: data ingestion, prompt generation, agent orchestration, evaluation

d. Key technologies and frameworks
– Python 3.x for orchestration and logic
– Shell scripting for experiment launch
– HTTP/REST calls to LLM endpoints
– JSON/CSV data formats for datasets and results
– human-eval framework for standard evaluation

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Scripts
    subgraph "Shell Scripts"
        RunAll((run.sh)):::scripts
        Gen((run_generation_task.sh)):::scripts
        Reason((run_reasoning_task.sh)):::scripts
    end

    RunAll --> Gen
    RunAll --> Reason
    Gen --> Main
    Reason --> Main

    %% Orchestrator
    Main["Orchestrator\nsrc/main.py"]:::module

    %% Data Layer
    subgraph "Data Layer"
        DL["DataLoader\nsrc/dataloader.py"]:::module
        DS[(Raw Datasets)]:::data
    end

    Main --> DL
    DL --> DS

    %% Task Modules
    subgraph "Task Modules"
        CT["Chess Task\nsrc/chess_task.py"]:::module
        GT["GSM Task\nsrc/gsm_task.py"]:::module
        MT["Math Task\nsrc/math_task.py"]:::module
        UT["MMLU Task\nsrc/mmlu_task.py"]:::module
        CodeT["Code Completion Task\nsrc/code_completion_task.py"]:::module
        PL["Prompt Library\nsrc/prompt_lib.py"]:::module
    end

    Main --> CT
    Main --> GT
    Main --> MT
    Main --> UT
    Main --> CodeT
    Main --> PL

    %% Agent Layer
    subgraph "Agent Layer"
        SA["Single-Agent\nsrc/agent.py"]:::module
        MA["Multi-Agent\nsrc/more_agent.py"]:::module
    end

    Main --> SA
    Main --> MA

    %% LLM Interface & External Service
    UI["LLM Interface\nsrc/utils.py"]:::service
    SA --> UI
    MA --> UI
    UI --> ES["External LLM Service"]:::external
    ES --> UI

    %% Evaluation Engine
    subgraph "Evaluation Engine"
        Eval["Evaluation\nsrc/evaluation.py"]:::module
        Cat["Categories\nsrc/categories.py"]:::module
    end

    Main --> Eval
    Cat --> Eval
    Eval --> RS[(Results Store)]:::data

    %% Click Events
    click RunAll "https://github.com/moreagentsisallyouneed/agentforest/blob/master/script/run.sh"
    click Gen "https://github.com/moreagentsisallyouneed/agentforest/blob/master/script/run_generation_task.sh"
    click Reason "https://github.com/moreagentsisallyouneed/agentforest/blob/master/script/run_reasoning_task.sh"
    click Main "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/main.py"
    click DL "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/dataloader.py"
    click DS "https://github.com/moreagentsisallyouneed/agentforest/tree/master/dataset/"
    click CT "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/chess_task.py"
    click GT "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/gsm_task.py"
    click MT "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/math_task.py"
    click UT "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/mmlu_task.py"
    click CodeT "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/code_completion_task.py"
    click PL "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/prompt_lib.py"
    click SA "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/agent.py"
    click MA "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/more_agent.py"
    click UI "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/utils.py"
    click Eval "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/evaluation.py"
    click Cat "https://github.com/moreagentsisallyouneed/agentforest/blob/master/src/categories.py"

    %% Styles
    classDef scripts fill:#e0f7fa,stroke:#006064,stroke-width:2px;
    classDef module fill:#e8f5e9,stroke:#1b5e20,stroke-width:2px;
    classDef service fill:#fff3e0,stroke:#bf360c,stroke-width:2px;
    classDef external fill:#e1bee7,stroke:#4a148c,stroke-dasharray:5 5,stroke-width:2px;
    classDef data fill:#fffde7,stroke:#f9a825,stroke-width:2px;
```
