# kagnlp/codegenerator architecture

> Architecture diagram of the GitHub repository kagnlp/codegenerator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-07). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kagnlp/codegenerator
- Repository: https://github.com/kagnlp/codegenerator

## Overview

##### Main Components
- Data Inputs & Datasets:
- Datasets directory (data) holds multiple benchmark datasets (APPS, HumanEval, MBPPEval, xCodeEval). These are the source problems on which the system operates.
- Supporting dataset conversion scripts exist in src/datasets (e.g., convert-apps-xcode.py).

- Core Agents:
- Planning Agent (from src/promptings, particularly the “Base”, “CoT”, “SelfPlanning” variations, etc.) is responsible for analyzing the problem description and generating a plan with exemplar guidance.
- Coding Agent (from src/models and possibly tied with promptings) which takes the generated plan and transforms it into executable code.
- Debugging Agent (covered in src/evaluations and src/utils) which simulates the execution, analyzes test case failures, and refines the code.

- Execution & Evaluation:
- Evaluation and simulation modules are located in src/evaluations and src/utils. These components use sample input/output test cases (and possibly integrate with external services like ExecEval) for verifying generated code.
- There is an API communication component (api_comm.py) to handle external execution environments. This supports docker-based execution as noted in the README.

- Results:
- The resulting outputs and logs are handled by components in src/results.

- Configurations & Constants:
- The project uses configuration constants under src/constants and environment configuration via .env.example.

##### Relationships and Interactions
- Input Flow:
- The user provides a problem description (and selects a specific dataset via command-line arguments).
- Data from benchmark datasets is loaded and pre-processed by the dataset components.

- Processing Flow:
- The Planning Agent takes in the problem and generates a plan (using strategies like exemplar-based planning and simulation verification).
- The Coding Agent uses the validated plan to produce executable code.
- The Debugging Agent receives the code along with execution logs and then simulates the code to identify errors and refine it.

- Output Flow:
- Results are captured, logged, and stored in the results module.
- There is also an integration point for evaluation against standard benchmarks, logging performance metrics (pass@1 scores, etc.).

- External and Auxiliary Services:
- Integration with external debuggers or runtime environments can be depicted (e.g., ExecEval via docker, with configuration set by modifications in api_comm.py).

##### Architectural Patterns and Design Principles
- Multi-Agent Pipeline:
- Sequential collaboration between the Planning, Coding, and Debugging agents, mirroring a human-like step-by-step approach.
- Modularity:
- Each agent is modularized (with specific modules under src/models, src/promptings, src/evaluations) making the system extensible.
- Simulation-Driven Verification:
- The design highlights simulation-based internal debugging, an approach central to CodeSim’s performance.

##### Technologies, Frameworks, and Libraries
- Python is the primary language, with dependencies listed in requirements.txt.
- Integration with external model providers like OpenAI is apparent (via src/models/OpenAI.py).
- YAML configuration files (e.g., limits_by_lang.yaml) indicate configuration-driven execution limits.
- Any LLM-based processing is embedded within the multi-agent system (seen in both promptings and evaluations modules).

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Inputs & Datasets
    subgraph "Data Inputs & Datasets"
        UI["User Input/Problem Description"]:::user
        Loader["Dataset Loader"]:::dataset
        Conversion["Dataset Conversion Script"]:::dataset
        Modules["Dataset Modules"]:::dataset
    end

    %% Core Agents
    subgraph "Core Agents"
        subgraph "Planning Agent"
            base["Base"]:::agent
            cot["Chain-of-Thought"]:::agent
            self["SelfPlanning"]:::agent
        end
        subgraph "Coding Agent"
            coding["Core Coding"]:::agent
            openai["OpenAI Integration"]:::agent
        end
        subgraph "Debugging Agent"
            eval["Evaluation"]:::agent
            utils["Debug Utilities"]:::agent
        end
    end

    %% Execution & Evaluation
    subgraph "Execution & Evaluation"
        External["Execution API (Docker/ExecEval)"]:::external
    end

    %% Results & Configurations
    subgraph "Results & Configurations"
        Results["Results & Logs"]:::results
        Config["Configurations & Constants"]:::config
        envConfig["Environment Config (.env.example)"]:::config
    end

    %% Data Flow Connections
    UI -->|"provides_problem"| Loader
    Loader -->|"loads_data"| Modules
    Loader -->|"runs_conversion"| Conversion
    Modules -->|"preprocess_data"| base
    Conversion -->|"preprocess_data"| base

    %% Agent Processing Flow
    base -->|"generates_plan"| coding
    cot -->|"generates_plan"| coding
    self -->|"generates_plan"| coding
    coding -->|"transforms_to_code"| eval
    openai -->|"enhances_code"| coding

    %% Debugging Feedback and Evaluation
    eval -->|"debugs_and_feedback"| coding
    eval -->|"evaluates"| Results
    eval -->|"calls_exec_api"| External
    External -->|"returns_results"| eval

    %% Configurations Support
    Config -->|"supports"| base
    Config -->|"supports"| coding
    Config -->|"supports"| eval
    envConfig -->|"env_support"| Config
    Results -->|"logs_outcome"| Config

    %% Click Events for Data Inputs & Datasets
    click Loader "https://github.com/kagnlp/codegenerator/tree/master/data/"
    click Conversion "https://github.com/kagnlp/codegenerator/blob/master/src/datasets/convert-apps-xcode.py"
    click Modules "https://github.com/kagnlp/codegenerator/tree/master/src/datasets/"

    %% Click Events for Planning Agent
    click base "https://github.com/kagnlp/codegenerator/blob/master/src/promptings/Base.py"
    click cot "https://github.com/kagnlp/codegenerator/blob/master/src/promptings/CoT.py"
    click self "https://github.com/kagnlp/codegenerator/blob/master/src/promptings/SelfPlanning.py"

    %% Click Events for Coding Agent
    click coding "https://github.com/kagnlp/codegenerator/tree/master/src/models/"
    click openai "https://github.com/kagnlp/codegenerator/blob/master/src/models/OpenAI.py"

    %% Click Events for Debugging Agent
    click eval "https://github.com/kagnlp/codegenerator/blob/master/src/evaluations/api_comm.py"
    click utils "https://github.com/kagnlp/codegenerator/tree/master/src/utils/"

    %% Click Event for External Execution API
    click External "https://github.com/kagnlp/codegenerator/blob/master/src/evaluations/api_comm.py"

    %% Click Events for Results & Configurations
    click Results "https://github.com/kagnlp/codegenerator/tree/master/src/results/"
    click Config "https://github.com/kagnlp/codegenerator/tree/master/src/constants/"
    click envConfig "https://github.com/kagnlp/codegenerator/blob/master/.env.example"

    %% Styles
    classDef user fill:#FFC0CB,stroke:#333,stroke-width:2px;
    classDef dataset fill:#B3CDE0,stroke:#333,stroke-width:2px;
    classDef agent fill:#C1E1C1,stroke:#333,stroke-width:2px;
    classDef external fill:#F4A460,stroke:#333,stroke-width:2px;
    classDef results fill:#D3D3D3,stroke:#333,stroke-width:2px;
    classDef config fill:#FFFACD,stroke:#333,stroke-width:2px;
```
