# ysz/recursive-llm architecture

> Architecture diagram of the GitHub repository ysz/recursive-llm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ysz/recursive-llm
- Repository: https://github.com/ysz/recursive-llm

## Overview

This repo is a **Python library / SDK** (not a deployed service) that implements **Recursive Language Models (RLM)** to let LLMs handle **unbounded / very large contexts** by **keeping the full context in a Python variable** and letting the model **interactively explore it via a restricted Python REPL**, including **recursive sub-calls** to cheaper models.

So the “system” you’re diagramming is primarily an **in-process architecture**:
- a user calls `RLM.complete()` / `RLM.acomplete()`
- the library orchestrates LLM calls (via **LiteLLM**) and safe local code execution (via **RestrictedPython**)
- the LLM iterates in a REPL loop until it emits `FINAL(answer)`

- **Context is never sent wholesale** to the provider
- The LLM sees **instructions + query**, and can “peek/search/slice” the in-memory context through **executed code**
- Recursive calls split context and solve subproblems

At runtime, the key loop is:

##### `core.py` initializes a REPL environment with
- `context` bound as a Python variable (the “unbounded context store”)
- helper functions, especially **`recursive_llm(...)`** that triggers nested RLM calls
##### `prompts.py` constructs the system prompt and “protocol”
- how to explore `context` via Python
- how to decide when to recurse
- requirement to finish with `FINAL(answer)`
##### `core.py` calls LiteLLM to reach an LLM provider
- root model: `model=...`
- optional recursive model: `recursive_model=...`
##### LLM returns a response containing either
- Python to execute (e.g., `context[:1000]`, regex search, partitioning)
- or the terminal `FINAL(...)`
##### `parser.py` detects whether `FINAL(...)` is present.
- If **FINAL**: return to user
- Else: `repl.py` executes the code (restricted), captures output, feeds results back into the next LLM step
##### The loop continues until
- `FINAL` found, or
- `max_iterations` exceeded, or
- `max_depth` exceeded (for recursion), etc.

Recursion path:
- When model executes `recursive_llm("subquery", context_slice)`, that creates a **nested orchestrator call** in `core.py`, usually using the **cheaper `recursive_model`**.

Async path:

##### Component details to explicitly annotate (RLM-specific, don’t omit)

- **Context Handling:** “Context stored in REPL variable; not appended to prompt”
- **Safety Boundary:** “RestrictedPython sandbox” around executing model code
- **Protocol:** “Model must output `FINAL(...)` to terminate”
- **Limits / Guards:** `max_iterations`, `max_depth` (and error path: “Max iterations exceeded”)
- **Model Split:** root model vs recursive model (cost optimization)
- **Provider Abstraction:** LiteLLM as the single integration surface (not direct OpenAI SDK usage)

## Diagram (Mermaid)

```mermaid
flowchart TD
  %% =========================
  %% A) Context / Ecosystem
  %% =========================
  subgraph "A) Ecosystem (System Context)"
    direction TB

    UA["User Application (Python)"]:::external
    ENV["Environment Variables / API Keys"]:::config
    RLM_SYS["RLM Library (this repo)"]:::library

    subgraph "External Dependencies"
      direction TB
      LITELLM["LiteLLM (Provider Abstraction)"]:::external
      PROV["LLM Providers (OpenAI/Anthropic/Azure/llama.cpp/...)"]:::cloud
      OLLAMA["Local Ollama Server (optional)"]:::cloud
      RP["RestrictedPython"]:::external
    end

    CTX[("In-memory Context\n(Python variable; not sent wholesale)")]:::datastore

    UA -->|"calls complete()/acomplete()"| RLM_SYS
    ENV -->|"secrets/config (e.g., OPENAI_API_KEY)"| RLM_SYS
    RLM_SYS <-.->|"binds/reads context variable"| CTX

    RLM_SYS -->|"LLM requests (prompt+observations)"| LITELLM
    LITELLM -->|"network calls"| PROV
    LITELLM -.->|"if model='ollama/...'"| OLLAMA

    RLM_SYS -->|"restricted code execution"| RP
  end

  %% =========================
  %% B) Container / Module
  %% =========================
  subgraph "B) In-Process Module Architecture (src/rlm)"
    direction TB

    INIT["__init__.py\nPublic API"]:::module
    CORE["core.py\nOrchestrator\n(max_iterations,max_depth,model,recursive_model)"]:::module
    PROMPTS["prompts.py\nPrompt Builder\n(protocol + FINAL(...))"]:::module
    PARSER["parser.py\nOutput Parser\n(extract FINAL / code)"]:::module
    REPL["repl.py\nREPL Executor\n(context var + helpers)\n(sequential exec)"]:::module
    TYPES["types.py\nTypes/Config DTOs"]:::types

    SANDBOX{{"Sandbox Boundary\n(RestrictedPython)"}}:::sandbox
    CTX2[("In-memory Context Store\n(bound as variable in REPL)")]:::datastore

    %% Public entrypoints
    INIT -->|"exports RLM.complete()/acomplete()"| CORE

    %% Core dependencies
    CORE -->|"builds system prompt + instructions"| PROMPTS
    CORE -->|"interprets model response (code|FINAL)"| PARSER
    CORE <-.->|"REPL loop (observations<->next prompt)"| REPL
    CORE -->|"LLM call (root model; async I/O possible)"| LITELLM

    %% Types shared
    CORE -->|"uses"| TYPES
    PROMPTS -->|"uses"| TYPES
    PARSER -->|"uses"| TYPES
    REPL -->|"uses"| TYPES
    INIT -->|"re-exports"| TYPES

    %% REPL internals
    REPL -->|"executes snippet"| SANDBOX
    SANDBOX -->|"safe execution"| RP
    REPL <-.->|"reads/slices/searches without sending full context"| CTX2

    %% Recursion path
    REPL -.->|"helper: recursive_llm(subquery,context_slice)"| CORE
  end

  %% =========================
  %% C) Sequence (Truth Source)
  %% =========================
  subgraph "C) Runtime Control Loop (Sequence)"
    direction TB

    subgraph "Lifelines"
      direction TB
      U2["User"]:::external
      C2["RLM Core (core.py)"]:::module
      P2["Prompt Builder (prompts.py)"]:::module
      L2["LLM (via LiteLLM)"]:::external
      PA2["Parser (parser.py)"]:::module
      R2["REPL Executor (repl.py)"]:::module
      SB2{{"RestrictedPython Sandbox"}}:::sandbox
      CXT3[("Context (in-memory variable)")]:::datastore
      RC2["Recursive RLM Core (nested)"]:::module
    end

    U2 -->|"complete(query,context,config)"| C2
    C2 -->|"init REPL env: bind context var + helpers"| R2
    R2 <-.->|"binds"| CXT3

    C2 -->|"build system prompt (protocol; must end FINAL(...))"| P2
    P2 -->|"instructions + query (no full context)"| C2

    C2 -.->|"loop until FINAL or max_iterations"| L2
    C2 -->|"prompt + observations"| L2
    L2 -->|"model response (code|FINAL)"| C2
    C2 -->|"parse response"| PA2

    PA2 -->|"FINAL? (yes/no)"| C2

    C2 -->|"if no FINAL: execute code in REPL"| R2
    R2 -->|"run snippet"| SB2
    SB2 -->|"restricted exec"| RP
    R2 -->|"observations (stdout/values/errors)"| C2

    C2 -.->|"continue loop (dashed control)"| L2

    %% Termination / guard rails
    C2 -->|"if FINAL: return answer"| U2
    C2 -->|"if max_iterations exceeded: error"| U2
    C2 -->|"if max_depth exceeded: error"| U2

    %% Recursion branch (triggered by executed code)
    R2 -.->|"recursive_llm(subquery,context_slice)"| RC2
    RC2 -->|"uses recursive_model (cheaper)"| L2
    RC2 -.->|"nested loop; returns sub-answer"| R2
  end

  %% =========================
  %% Examples / Tests (repo usage)
  %% =========================
  subgraph "Repo Usage (Examples / Tests)"
    direction TB
    EXS["examples/ (happy paths)"]:::folder
    TSTS["tests/ (verification)"]:::folder

    EX_BASIC["basic_usage.py"]:::example
    EX_TWO["two_models.py"]:::example
    EX_OLL["ollama_local.py"]:::example
    EX_LONG["long_document.py"]:::example
    EX_MULTI["multi_file.py"]:::example
    EX_CFG["custom_config.py"]:::example
    EX_EXTR["data_extraction.py"]:::example
    EX_DEMO["demo.py"]:::example
    EX_ENV["setup_env.py"]:::example

    T_CORE["test_core.py"]:::test
    T_INT["test_integration.py"]:::test
    T_PARSER["test_parser.py"]:::test
    T_REPL["test_repl.py"]:::test

    EXS --> EX_BASIC
    EXS --> EX_TWO
    EXS --> EX_OLL
    EXS --> EX_LONG
    EXS --> EX_MULTI
    EXS --> EX_CFG
    EXS --> EX_EXTR
    EXS --> EX_DEMO
    EXS --> EX_ENV

    TSTS --> T_CORE
    TSTS --> T_INT
    TSTS --> T_PARSER
    TSTS --> T_REPL

    EX_BASIC -->|"calls API"| INIT
    EX_TWO -->|"root vs recursive_model"| CORE
    EX_OLL -->|"model='ollama/...'"| LITELLM
    T_INT -->|"asserts core+repl+parser loop"| CORE
    T_PARSER -->|"asserts FINAL parsing"| PARSER
    T_REPL -->|"asserts restricted exec"| REPL
  end

  %% =========================
  %% Click Events (component mapping)
  %% =========================
  click INIT "https://github.com/ysz/recursive-llm/blob/main/src/rlm/__init__.py"
  click CORE "https://github.com/ysz/recursive-llm/blob/main/src/rlm/core.py"
  click REPL "https://github.com/ysz/recursive-llm/blob/main/src/rlm/repl.py"
  click PROMPTS "https://github.com/ysz/recursive-llm/blob/main/src/rlm/prompts.py"
  click PARSER "https://github.com/ysz/recursive-llm/blob/main/src/rlm/parser.py"
  click TYPES "https://github.com/ysz/recursive-llm/blob/main/src/rlm/types.py"
  click ENV "https://github.com/ysz/recursive-llm/blob/main/.env.example"

  click EX_BASIC "https://github.com/ysz/recursive-llm/blob/main/examples/basic_usage.py"
  click EX_CFG "https://github.com/ysz/recursive-llm/blob/main/examples/custom_config.py"
  click EX_EXTR "https://github.com/ysz/recursive-llm/blob/main/examples/data_extraction.py"
  click EX_DEMO "https://github.com/ysz/recursive-llm/blob/main/examples/demo.py"
  click EX_LONG "https://github.com/ysz/recursive-llm/blob/main/examples/long_document.py"
  click EX_MULTI "https://github.com/ysz/recursive-llm/blob/main/examples/multi_file.py"
  click EX_OLL "https://github.com/ysz/recursive-llm/blob/main/examples/ollama_local.py"
  click EX_ENV "https://github.com/ysz/recursive-llm/blob/main/examples/setup_env.py"
  click EX_TWO "https://github.com/ysz/recursive-llm/blob/main/examples/two_models.py"

  click T_CORE "https://github.com/ysz/recursive-llm/blob/main/tests/test_core.py"
  click T_INT "https://github.com/ysz/recursive-llm/blob/main/tests/test_integration.py"
  click T_PARSER "https://github.com/ysz/recursive-llm/blob/main/tests/test_parser.py"
  click T_REPL "https://github.com/ysz/recursive-llm/blob/main/tests/test_repl.py"

  %% =========================
  %% Styles
  %% =========================
  classDef module fill:#2D7FF9,stroke:#0B3D91,color:#ffffff,stroke-width:1.5px
  classDef types fill:#2563EB,stroke:#0B3D91,color:#ffffff,stroke-width:1.5px
  classDef library fill:#1D4ED8,stroke:#0B3D91,color:#ffffff,stroke-width:2px
  classDef datastore fill:#E5E7EB,stroke:#6B7280,color:#111827,stroke-width:1.5px
  classDef sandbox fill:#F59E0B,stroke:#B45309,color:#111827,stroke-width:2px
  classDef cloud fill:#7C3AED,stroke:#4C1D95,color:#ffffff,stroke-width:2px
  classDef external fill:#0EA5E9,stroke:#075985,color:#ffffff,stroke-width:1.5px
  classDef config fill:#64748B,stroke:#334155,color:#ffffff,stroke-width:1.5px
  classDef folder fill:#10B981,stroke:#065F46,color:#052e2b,stroke-width:1.5px
  classDef example fill:#34D399,stroke:#065F46,color:#052e2b,stroke-width:1.2px
  classDef test fill:#FB7185,stroke:#9F1239,color:#3b0a12,stroke-width:1.2px
```
