# salesforce/bolaa architecture

> Architecture diagram of the GitHub repository salesforce/bolaa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-28). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/salesforce/bolaa
- Repository: https://github.com/salesforce/bolaa

## Overview

##### Project Type and Purpose
- This repo implements a benchmarking framework (“BOLAA”) for LLM-augmented autonomous agents.
- It orchestrates six agent architectures (Zeroshot, ReAct, PlanAct, etc.) paired with different LLMs and evaluates them in two environments:
– HotPotQA question-answering
– Webshop navigation

##### Top-Level Components (based on file tree)
A. Orchestrators (CLI entrypoints)
- run_hotpotqaagent.py
- run_webagent.py
B. Agent Frameworks
- hotpotqa_run/ (agent definitions, environment wrappers, evaluation, data)
- web_run/ (same structure, specialized for web tasks)
C. Environment Implementations
- hotpotqa_run/hotpotqa_env.py + wikienv.py
- web_run/web_env.py
- webshop/ (full Flask-based webshop simulation, Selenium env, shop_agent)
D. LLM Interface
- hotpotqa_run/llms.py & web_run/llms.py (wrap calls to OpenAI or FastChat)
E. Utilities & Configuration
- config.py in each run folder
- pre_prompt.py, utils.py, wrappers.py
F. Test Suites & Data
- webshop/tests, hotpotqa_run/data/*.joblib

##### Interactions and Data Flow
1. CLI invokes the appropriate “run_*” script.
2. Script loads config → registers agent architecture from agent_arch.py.
3. Orchestrator loops through episodes:
a. Calls environment.reset() → returns initial observation.
b. Agent receives observation → generates action via llm interface.
c. Environment.step(action) → returns next observation & reward.
d. Repeat until done; record metrics.
4. Evaluator collects per-episode rewards, logs, and aggregates.
5. (Webshop only) Agent actions trigger browser interactions via Selenium in webshop/web_agent_site/envs.

##### Architectural Patterns and Design Principles
- Modular Environment Abstraction (Env API similar to OpenAI Gym)
- Strategy Pattern for agent architectures (each variant as a class in agent_arch.py)
- Adapter Pattern for LLM layer (llms.py abstracts OpenAI vs. local models)
- Separation of Concerns:
– CLI/orchestrator vs. agent logic vs. environment simulation vs. evaluation

##### Technologies & Frameworks to Highlight
- Python 3.10, Conda environment
- LangChain (for prompt orchestration)
- OpenAI API / FastChat
- Selenium + ChromeDriver (webshop env)
- Flask / Jinja2 (web_agent_site)
- Joblib (caching HotPotQA data)

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% HotPotQA Benchmark
    subgraph "HotPotQA Benchmark"
        direction TB
        run_hotpotqaagent["CLI Runner (HotPotQA)"]:::core
        hotpot_config["Config"]:::util
        hotpot_agent_arch["Agent Architectures"]:::core
        hotpot_preprompt["Pre-prompt Templates"]:::util
        hotpot_fs["Few-Shot Loader"]:::util
        hotpot_utils["Utilities & Wrappers"]:::util
        hotpot_llm["LLM Interface"]:::core
        LMServiceH((OpenAI/FastChat)):::external
        hotpot_env_hotpotqa["HotPotQA Env"]:::env
        hotpot_env_wiki["Wiki Env"]:::env
        hotpot_cache["Joblib Data Store"]:::data
        hotpot_evaluate["Evaluator & Metrics"]:::core

        run_hotpotqaagent --> hotpot_config
        run_hotpotqaagent --> hotpot_agent_arch
        hotpot_agent_arch --> hotpot_preprompt
        hotpot_agent_arch --> hotpot_fs
        hotpot_agent_arch --> hotpot_utils
        hotpot_agent_arch --> hotpot_llm
        hotpot_llm --> LMServiceH
        hotpot_agent_arch --> hotpot_env_hotpotqa
        hotpot_env_hotpotqa --> hotpot_env_wiki
        hotpot_env_wiki --> hotpot_cache
        hotpot_env_hotpotqa --> hotpot_evaluate
        hotpot_env_wiki --> hotpot_evaluate
        hotpot_env_hotpotqa --> hotpot_agent_arch
    end

    %% Webshop Benchmark
    subgraph "Webshop Benchmark"
        direction TB
        run_webagent["CLI Runner (Webshop)"]:::core
        web_config["Config"]:::util
        web_agent_arch["Agent Architectures"]:::core
        web_multi_arch["Multi-Agent Architectures"]:::core
        web_preprompt["Pre-prompt Templates"]:::util
        web_utils["Utilities"]:::util
        web_llm["LLM Interface"]:::core
        LMServiceW((OpenAI/FastChat)):::external
        web_env["Webshop Env"]:::env
        flask_app["Flask Webshop App"]:::env
        flask_templates["Jinja2 Templates"]:::env
        shop_agent["Shop Agent"]:::env
        shop_attrs["Attributes"]:::env
        engine["Engine Core"]:::env
        goal["Goal Logic"]:::env
        normalize["Normalize"]:::env
        selenium_env_site["Selenium Env (Site)"]:::env
        selenium_env_text["Selenium Env (Text)"]:::env
        web_evaluate["Evaluator & Metrics"]:::core
        run_env_site["Run Env Site Script"]:::util
        run_env_text["Run Env Text Script"]:::util

        run_webagent --> web_config
        run_webagent --> web_agent_arch
        web_agent_arch --> web_multi_arch
        web_agent_arch --> web_preprompt
        web_agent_arch --> web_utils
        web_agent_arch --> web_llm
        web_llm --> LMServiceW
        web_agent_arch --> web_env
        web_env --> flask_app
        flask_app --> flask_templates
        web_env --> shop_agent
        web_env --> shop_attrs
        web_env --> engine
        engine --> goal
        engine --> normalize
        web_env --> selenium_env_site
        web_env --> selenium_env_text
        selenium_env_site --> flask_app
        selenium_env_text --> flask_app
        web_env --> web_evaluate
        web_agent_arch --> web_evaluate
        run_env_site --> web_env
        run_env_text --> web_env
    end

    %% Click Events
    click run_hotpotqaagent "https://github.com/salesforce/bolaa/blob/main/run_hotpotqaagent.py"
    click hotpot_config "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/config.py"
    click hotpot_agent_arch "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/agent_arch.py"
    click hotpot_preprompt "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/pre_prompt.py"
    click hotpot_fs "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/fewshots.py"
    click hotpot_utils "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/utils.py"
    click hotpot_llm "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/llms.py"
    click hotpot_env_hotpotqa "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/hotpotqa_env.py"
    click hotpot_env_wiki "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/wikienv.py"
    click hotpot_cache "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/data/easy.joblib"
    click hotpot_evaluate "https://github.com/salesforce/bolaa/blob/main/hotpotqa_run/evaluate.py"

    click run_webagent "https://github.com/salesforce/bolaa/blob/main/run_webagent.py"
    click web_config "https://github.com/salesforce/bolaa/blob/main/web_run/config.py"
    click web_agent_arch "https://github.com/salesforce/bolaa/blob/main/web_run/agent_arch.py"
    click web_multi_arch "https://github.com/salesforce/bolaa/blob/main/web_run/multi_agent_arch.py"
    click web_preprompt "https://github.com/salesforce/bolaa/blob/main/web_run/pre_prompt.py"
    click web_utils "https://github.com/salesforce/bolaa/blob/main/web_run/utils.py"
    click web_llm "https://github.com/salesforce/bolaa/blob/main/web_run/llms.py"
    click web_env "https://github.com/salesforce/bolaa/blob/main/web_run/web_env.py"
    click web_evaluate "https://github.com/salesforce/bolaa/blob/main/web_run/evaluate.py"
    click flask_app "https://github.com/salesforce/bolaa/blob/main/webshop/web_agent_site/app.py"
    click flask_templates "https://github.com/salesforce/bolaa/tree/main/webshop/web_agent_site/templates/"
    click shop_agent "https://github.com/salesforce/bolaa/blob/main/webshop/web_agent_site/agents/shop_agent.py"
    click shop_attrs "https://github.com/salesforce/bolaa/tree/main/webshop/web_agent_site/attributes/"
    click engine "https://github.com/salesforce/bolaa/blob/main/webshop/web_agent_site/engine/engine.py"
    click goal "https://github.com/salesforce/bolaa/blob/main/webshop/web_agent_site/engine/goal.py"
    click normalize "https://github.com/salesforce/bolaa/blob/main/webshop/web_agent_site/engine/normalize.py"
    click selenium_env_site "https://github.com/salesforce/bolaa/blob/main/webshop/web_agent_site/envs/web_agent_site_env.py"
    click selenium_env_text "https://github.com/salesforce/bolaa/blob/main/webshop/web_agent_site/envs/web_agent_text_env.py"
    click run_env_site "https://github.com/salesforce/bolaa/blob/main/webshop/run_envs/run_web_agent_site_env.py"
    click run_env_text "https://github.com/salesforce/bolaa/blob/main/webshop/run_envs/run_web_agent_text_env.py"

    %% Styles
    classDef core fill:#a6cee3,stroke:#333,stroke-width:1px;
    classDef env fill:#b2df8a,stroke:#333,stroke-width:1px;
    classDef external fill:#fdbf6f,stroke:#333,stroke-width:1px;
    classDef util fill:#d3d3d3,stroke:#333,stroke-width:1px;
    classDef data fill:#999999,stroke:#333,stroke-width:1px;
```
