How ayanami2003/gate works
Overall Architecture – Identify and Distinguish the Layers
• Data Layer: Represent the “Datasets” directory. There are multiple datasets (DABench, MATH, TabMWP, date, textcraft) each containing a JSONL dataset along with related “prompt_lib” configurations. These serve as both training and testing input for various task-specific experiments.
• Core Engine / Agent System: This module is primarily implemented in the “agent” directory and related experiment directories. It consists of:
• Agent Components: Sub-modules in agent/action (base action, checkingagent_action, solvingagent_action, toolagent_action, etc.) and agent/agents.py that model the different behaviors. Also include agent/skillagent.py for skill-specific processing.
• Environment Integration: Found in agent/environment (code, env, notebook, utils) – these provide execution contexts and sandboxing for code execution or simulated environments.
• Supporting Modules: agent/llm.py, agent/memory.py, and agent/utils.py help manage LLM integration, state tracking, and common functionality.
• Tool Graph Module: The “toolgraph” directory contains core functionalities (embedding_model.py, net_sturcture.py) that indicate a graph-based representation. This module likely builds, updates, and maintains the tool graphs that the agent uses.
• Experimentation / Training and Evaluation: The “experiment” directory holds different experiments – for DABench, Date, MATH, TabMWP, and textcraft – along with corresponding prompt libraries which allow for testing different training configurations and LLM interactions.
• System Orchestration: The “run.py” script serves as the entry point for training/building the tool graph and directs the overall control flow via command-line arguments (task_type, dataset_path, toolkit_path, output_path, etc.).
Relationships and Data Flow
• That the user (or an automated scheduler) provides inputs (datasets and toolkit configurations) via command-line using run.py.
• The input is then directed to the Agent System where the various agent components (solving, checking, skill, etc.) interact with the environment modules.
• These agents communicate with the Tool Graph module to evolve the structure by leveraging the LLM (see agent/llm.py) for making inferences and decisions.
• Feedback loops exist where agents use memory (agent/memory.py) and environment feedback (agent/environment/*) to refine their actions.
• The experiments orchestrate multiple tasks where experiments in the “experiment” directory serve as dedicated pipelines for evaluation.
Architectural Patterns and Design Principles
• Modularity: The clear separation into subdirectories (agent, toolgraph, experiment, datasets) shows adherence to a modular design.
• Pluggability & Extensibility: The presence of multiple “prompt_lib” folders across datasets and experiments and separate agent actions suggest that the framework is built to support different task types with minimal reconfiguration.
• External Integration: Emphasize the connection with external LLMs and APIs (e.g., OpenAI, vllm deployment shown in the README).