# foundationagents/spo architecture

> Architecture diagram of the GitHub repository foundationagents/spo, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-09-08). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/foundationagents/spo
- Repository: https://github.com/foundationagents/spo

## Overview

##### Project type and purpose
- This is a Python-based prompt engineering tool (“SPO”) for self-supervised prompt optimization on large language models (LLMs).
- It can be driven via a CLI (`optimize.py`), a Python API (`components/optimizer.py`), or a Streamlit web UI (`app.py`).
- It reads iteration templates (`settings/*.yaml`), optional data (`data/*/*.jsonl`), and LLM configuration (`config/*.yaml`), then repeatedly calls LLMs for optimization, evaluation, and execution, producing optimized prompts and answers in a workspace directory.

##### File structure
- Top-level entry points:
- `optimize.py` (CLI runner)
- `app.py` (Streamlit UI)
- Core logic:
- `components/optimizer.py` – orchestrates optimization loops
- `components/evaluator.py` – judges LLM outputs
- LLM interface:
- `llm/async_llm.py` – async wrappers around LLM API calls
- `utils/llm_client.py` – high-level client managing optimize/evaluate/execute calls
- Configuration & templates:
- `config/` – sample LLM API settings
- `settings/` – iteration templates defining prompts, requirements, QA pairs
- Prompts:
- `prompts/evaluate_prompt.py`, `prompts/optimize_prompt.py` – prompt engineering helpers
- Data utilities:
- `utils/data_utils.py`, `utils/prompt_utils.py`, `utils/evaluation_utils.py`, `utils/load.py`
- Datasets:
- `data/*` – training/validation/test JSONL files for various benchmarks (MATH, BBH, GPQA, etc.)
- Requirements & assets:
- `requirements.txt`, static images under `assert/`, and `LICENSE`

##### README for additional insights
- Confirms three operation modes: Python API, CLI, Streamlit UI.
- Workflow:
- Initialize LLM settings → create `PromptOptimizer` → call `optimize()` → generate workspace with per‐round prompt/answers and `results.json`.
- Highlights external LLM services (Claude, GPT-4) as “judge” and “executor.”

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User Interfaces
    subgraph "User Interfaces"
        CLI["CLI Runner"]:::ui
        API["Python API Entrypoint"]:::ui
        UI["Streamlit Web UI"]:::ui
    end

    %% Core Orchestrator
    Optimizer["PromptOptimizer\n(core loop)"]:::core

    %% Config & Settings
    subgraph "Configuration & Settings"
        Config["LLM API Settings"]:::data
        Settings1["Navigate.yaml"]:::data
        Settings2["Poem.yaml"]:::data
    end

    %% Data Loader
    subgraph "Data Loader"
        DataUtils["Data Utils"]:::data
        Load["Load Module"]:::data
        Datasets["Datasets (JSONL)"]:::data
    end

    %% Prompt Builders
    subgraph "Prompt Builders"
        OptPrompt["optimize_prompt.py"]:::builder
        EvalPrompt["evaluate_prompt.py"]:::builder
    end

    %% Evaluation Utilities
    EvalUtils["Evaluation Utils"]:::util

    %% Prompt Utilities
    PromptUtils["Prompt Utils"]:::util

    %% LLM Interface
    subgraph "LLM Interface"
        LLMClient["LLM Client"]:::core
        AsyncLLM["Async LLM Adapter"]:::core
    end

    %% Evaluator
    Evaluator["Evaluator"]:::core

    %% External Service
    External["External LLM Service"]:::external

    %% Output Writer
    Workspace["Workspace (Outputs)"]:::storage

    %% Connections
    CLI -->|"invokes"| Optimizer
    API -->|"invokes"| Optimizer
    UI -->|"invokes"| Optimizer

    Config --> Optimizer
    Settings1 --> Optimizer
    Settings2 --> Optimizer

    DataUtils --> Optimizer
    Load --> Optimizer
    Datasets --> Optimizer

    Optimizer -->|"builds prompts"| OptPrompt
    Optimizer -->|"builds eval prompts"| EvalPrompt
    OptPrompt --> LLMClient
    EvalPrompt --> LLMClient

    Optimizer --> AsyncLLM
    LLMClient --> AsyncLLM
    AsyncLLM -->|"API calls"| External

    Optimizer --> Evaluator
    Evaluator --> EvalUtils
    Evaluator --> PromptUtils
    Evaluator --> LLMClient

    Optimizer --> Workspace

    %% Loop
    Optimizer -.->|"iterate"| Optimizer

    %% Click Events
    click CLI "https://github.com/foundationagents/spo/blob/main/optimize.py"
    click UI "https://github.com/foundationagents/spo/blob/main/app.py"
    click API "https://github.com/foundationagents/spo/blob/main/components/optimizer.py"
    click Optimizer "https://github.com/foundationagents/spo/blob/main/components/optimizer.py"
    click Evaluator "https://github.com/foundationagents/spo/blob/main/components/evaluator.py"
    click AsyncLLM "https://github.com/foundationagents/spo/blob/main/llm/async_llm.py"
    click LLMClient "https://github.com/foundationagents/spo/blob/main/utils/llm_client.py"
    click OptPrompt "https://github.com/foundationagents/spo/blob/main/prompts/optimize_prompt.py"
    click EvalPrompt "https://github.com/foundationagents/spo/blob/main/prompts/evaluate_prompt.py"
    click Config "https://github.com/foundationagents/spo/blob/main/config/config2.example.yaml"
    click Settings1 "https://github.com/foundationagents/spo/blob/main/settings/Navigate.yaml"
    click Settings2 "https://github.com/foundationagents/spo/blob/main/settings/Poem.yaml"
    click DataUtils "https://github.com/foundationagents/spo/blob/main/utils/data_utils.py"
    click Load "https://github.com/foundationagents/spo/blob/main/utils/load.py"
    click EvalUtils "https://github.com/foundationagents/spo/blob/main/utils/evaluation_utils.py"
    click PromptUtils "https://github.com/foundationagents/spo/blob/main/utils/prompt_utils.py"

    %% Styles
    classDef ui fill:#AED6F1,stroke:#1F618D,color:#1F618D
    classDef core fill:#85C1E9,stroke:#1B4F72,color:#1B4F72
    classDef data fill:#ABEBC6,stroke:#196F3D,color:#196F3D
    classDef builder fill:#F9E79F,stroke:#B9770E,color:#B9770E
    classDef util fill:#F5CBA7,stroke:#A04000,color:#A04000
    classDef external fill:#F8C471,stroke:#B03A2E,color:#B03A2E
    classDef storage fill:#D2B4DE,stroke:#512E5F,color:#512E5F
```
