# md-ashraful-pramanik/mapcoder architecture

> Architecture diagram of the GitHub repository md-ashraful-pramanik/mapcoder, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/md-ashraful-pramanik/mapcoder
- Repository: https://github.com/md-ashraful-pramanik/mapcoder

## Overview

##### Main Components
- Entry Point/Controller: Represented by src/main.py, which parses command-line arguments (selecting model, dataset, and strategy) and drives the whole pipeline.
- Environment & Configuration: The .env files (.env.example.azure, .env.example.openai) and requirements.txt that configure external services and dependencies.
- Data & Datasets: The “data” directory holds various benchmark datasets (HumanEval, MBPP, APPS, CodeContest, xCodeEval) used for evaluation. Within src/datasets, you have dataset loaders (APPSDataset, CodeContestDataset, etc.) and a factory (DatasetFactory) that abstracts different dataset types.
- Agent Modules (Prompting): Found in src/promptings, including distinct files for Analogical, CoT (Chain-of-Thought), Direct, MapCoder, SelfPlanning. These files encapsulate the multi-agent strategies – with each file generally corresponding to a specific agent in the overall workflow.
- Model Management: Under src/models, you have classes for different underlying LLM models (OpenAI, Gemini, etc.) and a ModelFactory for configuration and instantiation.
- Evaluations and Execution: The src/evaluations package includes modules for interacting with external APIs (api_comm.py), evaluating generated code, and executing unit tests (exec_outcome.py, func_evaluate.py). Additionally, there is integration with an external dockerized service (ExecEval) for competitive dataset execution.
- Utility Components: The src/utils directory with utility modules (jsonl parsing, token counting, parsing) that assist both evaluation and data handling tasks.
- Constants: The src/constants repository contains shared constants like language mappings and paths.

##### Relationships and Data Flow
- The main.py acts as the central coordinator: it reads configuration, sets up the environment, and triggers the data processing, model selection, and evaluation pipelines.
- The dataset components (in src/datasets) load and convert dataset information (also with conversion scripts like convert-apps-xcode.py) and funnel test problems into the evaluation pipeline.
- The model components, selected via ModelFactory, interact with the various prompting strategies. The prompting modules (in src/promptings) are arranged in a pipeline that mimics a human-like coding cycle:
- The Retrieval Agent first recollects similar, relevant coding examples.
- The Planning Agent then produces a step-by-step solution plan.
- The Coding Agent converts this plan into actual code.
- Finally, the Debugging Agent revises and repairs the code based on test feedback.
- The evaluation package executes, validates, and records results (storing outputs into the “final-results” directory) across multiple benchmarks. The API communication module (api_comm.py) hints at additional external interactions.
- Utility functions support JSON data handling and token counting to ensure that prompt lengths and outputs meet specific constraints.

##### Architectural Patterns and Principles
- Modular Design: The separation of major concerns into discrete directories (datasets, models, promptings, evaluations) demonstrates a clean multi-layered modular architecture.
- Factory and Abstraction Patterns: Both the DatasetFactory and the ModelFactory encapsulate instantiation details for various datasets and models.
- Pipeline-Oriented Flow: The sequential flow of the multi-agent cycle (retrieval → planning → coding → debugging) is a central architectural concept.
- Extensibility: Clear separation into folders (e.g., different prompting strategies) makes it easy to add new agents or modify behavior without impacting unrelated parts of the system.

- Nodes/Components with clear, descriptive labels: “Main Controller (src/main.py)”, “Dataset Loader (src/datasets, conversion scripts)”, “Retrieval Agent”, “Planning Agent”, “Coding Agent”, “Debugging Agent”, “Model Factory (src/models)”, “Evaluation Engine (src/evaluations)”, “Utilities & Constants”, “External Services (ExecEval, APIs)”.
- Annotations for design patterns employed (factories, pipeline sequence).
- A clear demarcation between configuration files, data/input (from the “data” directory), processing modules, and output (saved in “final-results”).

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Entry Point"
        MAIN["Main Controller (src/main.py)"]:::main
    end

    subgraph "Configuration & Environment"
        CONFIG1[".env.example.azure"]:::config
        CONFIG2[".env.example.openai"]:::config
        CONFIG3["requirements.txt"]:::config
    end

    subgraph "Data & Datasets"
        DATA["Data Directory"]:::data
        DS_LOADER["Dataset Loaders & Factory (src/datasets)"]:::data
        CONV1["convert-apps-xcode.py"]:::data
        CONV2["convert-cc-xcode.py"]:::data
    end

    subgraph "Prompting Agents"
        RETRIEVAL["Retrieval Agent (Analogical)"]:::agent
        PLANNING["Planning Agent (CoT)"]:::agent
        CODING["Coding Agent (Direct)"]:::agent
        DEBUGGING["Debugging Agent (SelfPlanning)"]:::agent
        PROMPT_FACTORY["Prompting Factory (src/promptings/PromptingFactory.py)"]:::agent
        MAPCODER["MapCoder Strategy (src/promptings/MapCoder.py)"]:::agent
    end

    subgraph "Model Management"
        MODEL_FACTORY["Model Factory (src/models/ModelFactory.py)"]:::model
        OPENAI["OpenAI Model (src/models/OpenAI.py)"]:::model
        GEMINI["Gemini Model (src/models/Gemini.py)"]:::model
        BASE_MODEL["Base Model (src/models/Base.py)"]:::model
    end

    subgraph "Evaluation & Execution"
        EVALUATION_ENGINE["Evaluation Engine (src/evaluations)"]:::evaluation
        API_COMM["API Comm (src/evaluations/api_comm.py)"]:::evaluation
        EVALUATE["Evaluate Module (evalute.py)"]:::evaluation
        EXEC_OUTCOME["Execution Outcome (exec_outcome.py)"]:::evaluation
        FUNC_EVAL["Function Evaluation (func_evaluate.py)"]:::evaluation
        EXEC_UTILS["Executor Utils (executor_utils.py)"]:::evaluation
        RES_LIMIT["Resource Limit (resource_limit.py)"]:::evaluation
        LIMITS_YAML["Lang Limits (limits_by_lang.yaml)"]:::evaluation
        FINAL_RESULTS["Final Results Directory"]:::evaluation
    end

    subgraph "Utilities & Constants"
        UTIL["Utilities (src/utils)"]:::utils
        CONSTANTS["Constants (src/constants)"]:::constant
    end

    subgraph "External Services"
        EXTERNAL_API["External APIs"]:::external
        DOCKER["Dockerized ExecEval Service"]:::external
    end

    MAIN -->|"reads"| CONFIG1
    MAIN -->|"reads"| CONFIG2
    MAIN -->|"reads"| CONFIG3
    MAIN -->|"loads data from"| DATA
    MAIN -->|"invokes loaders in"| DS_LOADER
    DS_LOADER -->|"uses converters"| CONV1
    DS_LOADER -->|"uses converters"| CONV2

    MAIN -->|"selects model from"| MODEL_FACTORY
    MODEL_FACTORY -->|"instantiates"| OPENAI
    MODEL_FACTORY -->|"instantiates"| GEMINI
    MODEL_FACTORY -->|"instantiates"| BASE_MODEL

    MAIN -->|"triggers"| RETRIEVAL
    RETRIEVAL -->|"feeds into"| PLANNING
    PLANNING -->|"feeds into"| CODING
    CODING -->|"feeds into"| DEBUGGING

    RETRIEVAL -.->|"uses"| PROMPT_FACTORY
    PLANNING -.->|"uses"| PROMPT_FACTORY
    CODING -.->|"uses"| MAPCODER
    DEBUGGING -.->|"uses"| MAPCODER

    DEBUGGING -->|"sends output to"| EVALUATION_ENGINE
    EVALUATION_ENGINE -->|"calls"| API_COMM
    EVALUATION_ENGINE -->|"processes via"| EVALUATE
    EVALUATION_ENGINE -->|"processes via"| EXEC_OUTCOME
    EVALUATION_ENGINE -->|"processes via"| FUNC_EVAL
    EVALUATION_ENGINE -->|"utilizes"| EXEC_UTILS
    EVALUATION_ENGINE -->|"applies"| RES_LIMIT
    RES_LIMIT -->|"based_on"| LIMITS_YAML
    EVALUATION_ENGINE -->|"stores results in"| FINAL_RESULTS

    DS_LOADER -->|"supported_by"| UTIL
    EVALUATION_ENGINE -->|"supported_by"| UTIL
    MAIN -->|"references"| CONSTANTS

    EVALUATION_ENGINE -.->|"integrates"| DOCKER
    EVALUATION_ENGINE -.->|"calls"| EXTERNAL_API

    classDef main fill:#f9e79f,stroke:#b7950b,stroke-width:2px;
    classDef config fill:#aed6f1,stroke:#2471a3,stroke-width:2px;
    classDef data fill:#a3e4d7,stroke:#16a085,stroke-width:2px;
    classDef agent fill:#f5cba7,stroke:#d35400,stroke-width:2px;
    classDef model fill:#d7bde2,stroke:#8e44ad,stroke-width:2px;
    classDef evaluation fill:#fadbd8,stroke:#e74c3c,stroke-width:2px;
    classDef utils fill:#d5f5e3,stroke:#27ae60,stroke-width:2px;
    classDef constant fill:#fcf3cf,stroke:#f1c40f,stroke-width:2px;
    classDef external fill:#fdebd0,stroke:#e67e22,stroke-width:2px;

    click MAIN "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/main.py"
    click CONFIG1 "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/.env.example.azure"
    click CONFIG2 "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/.env.example.openai"
    click CONFIG3 "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/requirements.txt"
    click DATA "https://github.com/md-ashraful-pramanik/mapcoder/tree/master/data"
    click DS_LOADER "https://github.com/md-ashraful-pramanik/mapcoder/tree/master/src/datasets"
    click CONV1 "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/datasets/convert-apps-xcode.py"
    click CONV2 "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/datasets/convert-cc-xcode.py"
    click RETRIEVAL "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/promptings/Analogical.py"
    click PLANNING "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/promptings/CoT.py"
    click CODING "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/promptings/Direct.py"
    click DEBUGGING "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/promptings/SelfPlanning.py"
    click PROMPT_FACTORY "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/promptings/PromptingFactory.py"
    click MAPCODER "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/promptings/MapCoder.py"
    click MODEL_FACTORY "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/models/ModelFactory.py"
    click OPENAI "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/models/OpenAI.py"
    click GEMINI "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/models/Gemini.py"
    click BASE_MODEL "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/models/Base.py"
    click API_COMM "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/evaluations/api_comm.py"
    click EVALUATE "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/evaluations/evalute.py"
    click EXEC_OUTCOME "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/evaluations/exec_outcome.py"
    click FUNC_EVAL "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/evaluations/func_evaluate.py"
    click EXEC_UTILS "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/evaluations/executor_utils.py"
    click RES_LIMIT "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/evaluations/resource_limit.py"
    click LIMITS_YAML "https://github.com/md-ashraful-pramanik/mapcoder/blob/master/src/evaluations/limits_by_lang.yaml"
    click FINAL_RESULTS "https://github.com/md-ashraful-pramanik/mapcoder/tree/master/final-results"
    click UTIL "https://github.com/md-ashraful-pramanik/mapcoder/tree/master/src/utils"
    click CONSTANTS "https://github.com/md-ashraful-pramanik/mapcoder/tree/master/src/constants"
```
