# salt-nlp/dylan architecture

> Architecture diagram of the GitHub repository salt-nlp/dylan, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/salt-nlp/dylan
- Repository: https://github.com/salt-nlp/dylan

## Overview

##### Main Components
- Query / Input Module:
– Represents the initial task/query entry point. This is best highlighted by the demo (code/demo/run_DyLAN.py) where users input queries.
- Dynamic LLM-Agent Network Core:
– The “LLMLP” or LLM Neuron components. These are implemented in files named “LLMLP.py” and “LLM_Neuron.py” across the various dataset-specific directories (HumanEval, MATH, MMLU, demo).
– This core is responsible for the dynamic selection of agents (using an “Agent Importance Score”) and iterative communication among agents.
- Agent Team Optimization Module:
– This is the part of the system that evaluates and selects the optimal set of LLM agents based on their contributions. It is reflected in scripts like “anal_imp.sh” and the underlying algorithm in the code.
- Data/Evaluation Modules:
– Separate components exist for each dataset domain like MATH, MMLU, and HumanEval. Each subfolder contains scripts for evaluation, experiment setup, and result logging.
- External Integration:
– The system uses external LLM APIs (e.g., OpenAI’s API). This external dependency is signaled by the need to set an “OPENAI_API_KEY” in a number of scripts.
- Experiment and Logging Modules:

##### Relationships and Data Flow
- The user input (query) is passed to the Demo module, which then relays the query into the LLM-Agent network.
- The Agent Team Optimization module continuously monitors the contributions of each agent. Its output (via calculated “Agent Importance Scores”) then feeds back into selecting or pruning agents in subsequent rounds.
- At the evaluation stage, results are collected and compared against benchmark datasets in the MATH, HumanEval, and MMLU folders. These results are logged within the “exp” folder.

##### Highlighting Architectural Patterns and Design Principles
- A modular, layered architecture where each dataset (or experimental task) is treated as a separate “branch” or “instance” of the core network.
- Iterative and multi-round nature of the agent communication along with the early-stopping mechanism to enhance efficiency.
- Use of unsupervised metrics (Agent Importance Score) for optimization in the design.

##### Relevant Technologies, Frameworks, and Libraries
- Python (as the primary language)
- OpenAI LLM API integration (via external HTTP API calls)
- Shell scripts to run experiments (e.g., exp_*.sh)
- Standard libraries used in the project (modules like prompt_lib and utils suggest helper functions for queries and data processing)

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Input/Interface Layer
    Input["Demo Module (run_DyLAN.py)"]:::input

    %% LLM-Agent Network Core Layer
    subgraph "LLM-Agent Network Core"
        direction TB
        %% HumanEval group
        subgraph "HumanEval"
            HE1["LLM_Neuron"]:::core
            HE2["CoLLMLP"]:::core
        end
        %% MMLU group
        subgraph "MMLU"
            ML1["LLMLP"]:::core
            ML2["LLM_Neuron"]:::core
        end
        %% Demo group
        subgraph "Demo"
            D1["LLMLP"]:::core
            D2["LLM_Neuron"]:::core
        end
        %% MATH group
        subgraph "MATH"
            M1["llmlp_gen_math_listwise_cot"]:::core
            M2["llmlp_gen_math_listwise_deeper_markov"]:::core
            M3["llmlp_gen_mmlu_listwise"]:::core
        end
    end

    %% Agent Team Optimization Module
    Optimizer["Agent Team Optimizer (anal_imp.sh)"]:::optimizer

    %% External Integration
    External["External LLM API"]:::external

    %% Data/Evaluation Modules
    subgraph "Data/Evaluation Modules"
        direction TB
        HE_eval["HumanEval Eval (exp_human_eval.sh)"]:::evaluation
        MATH_eval1["MATH Eval (eval_math.py)"]:::evaluation
        MATH_eval2["MATH Eval (exp_math.sh)"]:::evaluation
        MATH_eval3["MATH Eval (exp_math_complex.sh)"]:::evaluation
        MATH_eval4["MATH Eval (eval_mmlu.py)"]:::evaluation
        MMLU_eval1["MMLU Eval (exp_mmlu.sh)"]:::evaluation
        MMLU_eval2["MMLU Eval (exp_mmlu_allsubset.sh)"]:::evaluation
    end

    %% Experiment and Logging Module
    Logging["Experiment & Logging (exp/)"]:::logging

    %% Connections
    Input -->|"sends query"| HE1
    %% Iterative communication among agents in the core
    HE1 -->|"iterative"| ML1
    ML1 -->|"iterative"| D1
    D1 -->|"iterative"| M1
    M1 -->|"iterative"| HE1

    %% Agent Team Optimization feedback
    HE2 -->|"monitors"| Optimizer
    Optimizer -->|"feedback"| HE1

    %% External LLM API integration from Demo group
    D2 -->|"calls"| External

    %% Data flow to Evaluation Modules and Logging
    HE1 -->|"outputs"| HE_eval
    HE_eval -->|"logs results"| Logging

    %% Additional connections from core to Evaluation Modules (for MATH and MMLU)
    M1 -->|"evaluates"| MATH_eval1
    M2 -->|"evaluates"| MATH_eval2
    M3 -->|"evaluates"| MATH_eval3
    ML2 -->|"evaluates"| MMLU_eval1
    ML2 -->|"evaluates"| MMLU_eval2

    %% Styles
    classDef input fill:#E3F2FD,stroke:#1E88E5,stroke-width:2px;
    classDef core fill:#FFF3E0,stroke:#F57C00,stroke-width:2px;
    classDef optimizer fill:#E8F5E9,stroke:#43A047,stroke-width:2px;
    classDef external fill:#FFFDE7,stroke:#F9A825,stroke-width:2px;
    classDef evaluation fill:#F3E5F5,stroke:#8E24AA,stroke-width:2px;
    classDef logging fill:#FCE4EC,stroke:#D81B60,stroke-width:2px;

    %% Click Events
    click Input "https://github.com/salt-nlp/dylan/blob/main/code/demo/run_DyLAN.py"
    click HE1 "https://github.com/salt-nlp/dylan/blob/main/code/HumanEval/LLM_Neuron.py"
    click HE2 "https://github.com/salt-nlp/dylan/blob/main/code/HumanEval/CoLLMLP.py"
    click ML1 "https://github.com/salt-nlp/dylan/blob/main/code/MMLU/LLMLP.py"
    click ML2 "https://github.com/salt-nlp/dylan/blob/main/code/MMLU/LLM_Neuron.py"
    click D1 "https://github.com/salt-nlp/dylan/blob/main/code/demo/LLMLP.py"
    click D2 "https://github.com/salt-nlp/dylan/blob/main/code/demo/LLM_Neuron.py"
    click M1 "https://github.com/salt-nlp/dylan/blob/main/code/MATH/llmlp_gen_math_listwise_cot.py"
    click M2 "https://github.com/salt-nlp/dylan/blob/main/code/MATH/llmlp_gen_math_listwise_deeper_markov.py"
    click M3 "https://github.com/salt-nlp/dylan/blob/main/code/MATH/llmlp_gen_mmlu_listwise.py"
    click Optimizer "https://github.com/salt-nlp/dylan/blob/main/code/MMLU/anal_imp.sh"
    click HE_eval "https://github.com/salt-nlp/dylan/blob/main/code/HumanEval/exp_human_eval.sh"
    click MATH_eval1 "https://github.com/salt-nlp/dylan/blob/main/code/MATH/eval_math.py"
    click MATH_eval2 "https://github.com/salt-nlp/dylan/blob/main/code/MATH/exp_math.sh"
    click MATH_eval3 "https://github.com/salt-nlp/dylan/blob/main/code/MATH/exp_math_complex.sh"
    click MATH_eval4 "https://github.com/salt-nlp/dylan/blob/main/code/MATH/eval_mmlu.py"
    click MMLU_eval1 "https://github.com/salt-nlp/dylan/blob/main/code/MMLU/exp_mmlu.sh"
    click MMLU_eval2 "https://github.com/salt-nlp/dylan/blob/main/code/MMLU/exp_mmlu_allsubset.sh"
    click Logging "https://github.com/salt-nlp/dylan/tree/main/exp/"
```
