# bwz96sco/or_llm_agent architecture

> Architecture diagram of the GitHub repository bwz96sco/or_llm_agent, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bwz96sco/or_llm_agent
- Repository: https://github.com/bwz96sco/or_llm_agent

## Overview

##### Project type and purpose
- This is an AI-agent framework (open-source tool) that orchestrates multiple LLM-based sub-agents to convert natural-language OR problems into mathematical models, generate solver code, debug it, execute via Gurobi, and evaluate results.

##### File structure
- Top-level scripts:
– or_llm_eval.py / or_llm_eval_async_resilient.py / or_llm_show.py: evaluation entry-points
– mcp_server.py: optional local microservice for multi-choice problems (MCP)
- Data layer (data/): dataset JSONs, processing scripts, conversion utilities
- Configuration: .env.example, python-version, requirements.txt, pyproject.toml
- Utilities: utils.py for shared helper functions

##### README for architecture insights
- Three sequential stages (sub-agents):
a. Mathematical Modeling Agent
b. Code Generation Agent
c. Debugging Agent
- Orchestrator runs each in turn, using LLM APIs (OpenAI, Claude, DeepSeek)
- Uses Gurobi Optimizer to solve generated code
- Batch and asynchronous evaluation supported

a. **User / Client**: supplies natural-language problem descriptions or dataset
b. **Orchestrator** (scripts/orchestrator module)
– Parses CLI args
– Loads problem input
– Invokes sub-agents sequentially or in parallel (async)
c. **Modeling Agent**
– Calls LLM API to produce a mathematical model (e.g., in Pyomo or Gurobi API)
d. **Code Generation Agent**
– Takes the model spec, prompts LLM to generate executable Python code
e. **Debugging Agent**
– Submits generated code to LLM for fix suggestions on errors
f. **Execution Engine**
– Runs the final code locally
– Interfaces with Gurobi Solver
g. **Data Store**
– Input datasets (data/datasets/*.json)
– Processed intermediate files
h. **Evaluation & Reporting**
– or_llm_eval*.py collects solutions, compares against ground truth, produces metrics
i. **External Services**
– LLM Providers (OpenAI, Claude, DeepSeek via API_BASE)
– Gurobi Optimizer

##### Relationships and interactions
- User → Orchestrator: CLI invocation or script call
- Orchestrator → Modeling Agent → LLM API → returns model spec
- Modeling Agent → Code Generation Agent → LLM API → returns code
- Code Generation Agent → Debugging Agent (if enabled) → LLM API → returns patched code
- Debugged code → Execution Engine → Gurobi → solution
- Solution → Evaluation module → metrics / logs → User
- Data Store is read by Orchestrator and agents; results stored or logged

##### Architectural patterns & design principles
- **Pipeline / Agent Pattern**: each sub-agent is a self-contained component with a single responsibility
- **Prompts as Interfaces**: LLMs abstracted as “services” behind prompt wrappers
- **Async / Batch Processing**: evaluation scripts support concurrency for throughput
- **Modular Data Processing**: scripts under data/ handle conversion, indexing, and metrics

##### Technologies & frameworks
- Python 3.8+
- LLM APIs: OpenAI (text-davinci family), Claude, DeepSeek
- Gurobi Python API (solver)
- Pytest or custom tests under data/ for dataset preparation
- Shell scripts (run_eval_batch_agent.sh) for orchestration

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Entry Scripts invoked by User
    User["User / Client"]:::external
    subgraph "Entry Scripts (Local Runtime)"
        Eval["or_llm_eval.py"]:::orchestrator
        EvalAsync["or_llm_eval_async_resilient.py"]:::orchestrator
        Show["or_llm_show.py"]:::orchestrator
        Batch["run_eval_batch_agent.sh"]:::orchestrator
    end

    %% Core Pipeline
    subgraph "Core Pipeline (Local Runtime)"
        Modeling["Modeling Agent"]:::modeling
        CodeGen["Code Generation Agent"]:::codegen
        Debugging["Debugging Agent"]:::debugging
        Execution["Execution Engine"]:::execution
    end

    %% Data Store and Processing
    subgraph "Data Store & Processing (Local Runtime)"
        DataStore["Data Store"]:::datastore
        subgraph "Data Processing & Conversion"
            P1["process_dataset.py"]:::datastore
            P2["process_complexor_dataset.py"]:::datastore
            P3["process_lpwp_dataset.py"]:::datastore
            C1["convert_default_to_industryOR_format.py"]:::datastore
            C2["convert_industryOR_to_default_format.py"]:::datastore
        end
    end

    %% External Services
    subgraph "External Services"
        LLM["LLM Providers\n(OpenAI, Claude, DeepSeek)"]:::external
        Gurobi["Gurobi Optimizer"]:::external
        MCP["MCP Microservice"]:::external
    end

    %% Flow
    User -->|invokes| Eval
    User -->|invokes| EvalAsync
    User -->|invokes| Show
    User -->|runs| Batch

    Eval -->|parses & orchestrates| Modeling
    EvalAsync -->|parses & orchestrates| Modeling
    Show -->|parses & orchestrates| Modeling
    Batch -->|parses & orchestrates| Modeling

    Modeling -->|"prompt + model spec"| LLM
    LLM -->|"model spec"| Modeling

    Modeling -->|model spec| CodeGen
    CodeGen -->|"prompt + model spec"| LLM
    LLM -->|"generated code"| CodeGen

    CodeGen -->|code| Debugging
    Debugging -->|"prompt + error logs"| LLM
    LLM -->|"patched code"| Debugging

    Debugging -->|final code| Execution
    Execution -->|"executes code"| Gurobi
    Gurobi -->|"solution"| Execution

    %% Data interactions
    Eval -->|reads| DataStore
    Modeling -->|reads| DataStore
    CodeGen -->|reads| DataStore
    Debugging -->|reads| DataStore
    Execution -->|logs results to| DataStore

    DataStore -->|uses scripts| P1 & P2 & P3 & C1 & C2

    %% External MCP service
    Eval -->|optional MCP call| MCP

    %% Click Events
    click Eval "https://github.com/bwz96sco/or_llm_agent/blob/main/or_llm_eval.py"
    click EvalAsync "https://github.com/bwz96sco/or_llm_agent/blob/main/or_llm_eval_async_resilient.py"
    click Show "https://github.com/bwz96sco/or_llm_agent/blob/main/or_llm_show.py"
    click Batch "https://github.com/bwz96sco/or_llm_agent/blob/main/run_eval_batch_agent.sh"
    click P1 "https://github.com/bwz96sco/or_llm_agent/blob/main/data/process_dataset.py"
    click P2 "https://github.com/bwz96sco/or_llm_agent/blob/main/data/process_complexor_dataset.py"
    click P3 "https://github.com/bwz96sco/or_llm_agent/blob/main/data/process_lpwp_dataset.py"
    click C1 "https://github.com/bwz96sco/or_llm_agent/blob/main/data/convert_default_to_industryOR_format.py"
    click C2 "https://github.com/bwz96sco/or_llm_agent/blob/main/data/convert_industryOR_to_default_format.py"
    click DataStore "https://github.com/bwz96sco/or_llm_agent/tree/main/data/datasets/"
    click MCP "https://github.com/bwz96sco/or_llm_agent/blob/main/MCP/mcp_server.py"

    %% Styling
    classDef modeling fill:#cfe2f3,stroke:#036; 
    classDef codegen fill:#d9ead3,stroke:#274e13; 
    classDef debugging fill:#fce5cd,stroke:#b45f06; 
    classDef execution fill:#d9d2e9,stroke:#5b0f6f; 
    classDef datastore fill:#ffe599,stroke:#b06; 
    classDef orchestrator fill:#f4cccc,stroke:#a40000; 
    classDef external fill:#d0e0e3,stroke:#375a7f;
```
