# au-clan/cachesaver architecture

> Architecture diagram of the GitHub repository au-clan/cachesaver, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-05-30). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/au-clan/cachesaver
- Repository: https://github.com/au-clan/cachesaver

## Overview

##### Project type and purpose
- Cachesaver is a Python-based, open-source caching library specialized for large language model (LLM) experiments.
- Its goal is to reduce cost, improve reproducibility, and streamline debugging of batch LLM calls by providing multi-response caching, usage tracking, deduplication, and race-condition prevention.

##### File structure
- Top-level directories:
- src/ – core library code
­– algorithms/ (FOA, GOT, RAP, React, ToT)
­– models/ (API wrappers for OpenAI, Together, Groq, “online” dummy, QROQ)
­– tasks/ (task definitions for game24, hle, hotpotqa, humaneval, logiqa, matharena, scibench, sonnetwriting)
­– typedefs.py, utils.py (shared types and utilities)
- scripts/ – experiment drivers and pipelines
­– ablations/, correctness/, frameworks/, hyper_search/, motivation/, namespace/, old/, perms/, single/
- datasets/ – compressed datasets for each task
- examples/ – Jupyter notebooks demonstrating usage
- figures/ & prompts.md – experiment artifacts and prompt templates
- tests/ – pytest suite covering models, environments, benchmarks
- Config & build: requirements.txt, pytest.ini, API_proposal.md

##### README for additional insights
- Describes async, batching, deduplication, multi-response per query, statistical integrity.
- Mentions an external “cachesaver-core” package dependency.
- Emphasizes extensibility (multiple LLM API plugins) and experiment reproducibility.

##### Main components & their relationships
a. Core library (src/)
­– Cache Manager: orchestrates storing/fetching of query–response pairs, enforces deduplication and sampling policies
­– Algorithms: modules (foa.py, got.py, rap.py, react.py, tot_*.py) implementing different retrieval-oriented strategies
­– Task Definitions: environment, state, prompts, agents, and benchmark logic for each supported task
­– Model Adapters: asynchronous API wrappers to external LLM services (OpenAI, Together, Groq, QROQ, dummy online)
­– Utilities & Typedefs: shared helper functions, type definitions, exception handling
b. Experiment runner (scripts/)
­– Drives a sequence: load dataset → choose task → select algorithm → invoke library → collect & dump results
­– Subfolders for different experiment types (ablations, correctness checks, hyperparameter search, etc.)
c. Data storage
­– datasets/: raw inputs
­– local cache (configured by user or default file-based/DB backend via cachesaver-core)
­– output artifacts (figures/, examples/)
d. External services
­– LLM providers (via network calls)
­– User’s local filesystem / optional DB for persistent caching

##### Architectural patterns & design principles
- Modular plugin architecture: easily add new tasks, algorithms, or LLM adapters
- Async event loop: high-throughput, non-blocking API calls and batching
- Separation of concerns: caching engine decoupled from algorithm logic and from model interfaces
- Experiment reproducibility: scripts parameterize datasets, algorithms, and sampling seeds

## Diagram (Mermaid)

```mermaid
flowchart TB
  %% Experiment Drivers Layer
  subgraph "Experiment Drivers"
    direction TB
    driver["Experiment Runner"]:::drivers
    scripts_abl["Ablations"]:::drivers
    scripts_correctness["Correctness"]:::drivers
    scripts_frameworks["Frameworks"]:::drivers
    scripts_hyper["Hyper Search"]:::drivers
    scripts_motivation["Motivation"]:::drivers
    scripts_namespace["Namespace"]:::drivers
    scripts_old["Old"]:::drivers
    scripts_perms["Perms"]:::drivers
    scripts_single["Single"]:::drivers
  end

  %% Core Library Layer
  subgraph "Cachesaver Core Library"
    direction TB
    CacheMgr["Cache Manager"]:::internal

    subgraph "Algorithms"
      direction TB
      foa["FOA"]:::internal
      got["GOT"]:::internal
      rap["RAP"]:::internal
      react["ReAct"]:::internal
      tot_bfs["ToT_BFS"]:::internal
      tot_dfs["ToT_DFS"]:::internal
    end

    subgraph "Task Engine"
      direction TB
      t_game24["game24"]:::internal
      t_hle["hle"]:::internal
      t_hotpotqa["hotpotqa"]:::internal
      t_humaneval["humaneval"]:::internal
      t_logiqa["logiqa"]:::internal
      t_matharena["matharena"]:::internal
      t_scibench["scibench"]:::internal
      t_sonnet["sonnetwriting"]:::internal
    end

    subgraph "Model Adapters"
      direction TB
      m_anthropic["Anthropic"]:::internal
      m_api["OpenAI/API"]:::internal
      m_online["Online Dummy"]:::internal
      m_qroq["QROQ"]:::internal
    end

    subgraph "Utilities & Typedefs"
      direction TB
      utils["utils.py"]:::internal
      typedefs["typedefs.py"]:::internal
    end
  end

  %% External Dependencies Layer
  subgraph "External Dependencies"
    direction TB
    llm["LLM APIs"]:::external
    storage["File/DB Storage"]:::external
  end

  %% Data Storage & Artifacts
  subgraph "Data Storage & Artifacts"
    direction TB
    datasets["datasets"]:::storage
    figures["figures"]:::storage
    examples["examples"]:::storage
  end

  %% Connections
  driver -->|invokes| t_game24
  scripts_abl --> driver
  scripts_correctness --> driver
  scripts_frameworks --> driver
  scripts_hyper --> driver
  scripts_motivation --> driver
  scripts_namespace --> driver
  scripts_old --> driver
  scripts_perms --> driver
  scripts_single --> driver

  t_game24 -->|uses| foa
  t_hle -->|uses| foa
  t_hotpotqa -->|uses| foa
  t_humaneval -->|uses| foa
  t_logiqa -->|uses| foa
  t_matharena -->|uses| foa
  t_scibench -->|uses| foa
  t_sonnet -->|uses| foa

  foa -->|checks cache| CacheMgr
  got -->|checks cache| CacheMgr
  rap -->|checks cache| CacheMgr
  react -->|checks cache| CacheMgr
  tot_bfs -->|checks cache| CacheMgr
  tot_dfs -->|checks cache| CacheMgr

  CacheMgr -->|forwards miss| m_api
  CacheMgr -->|forwards miss| m_anthropic
  CacheMgr -->|forwards miss| m_online
  CacheMgr -->|forwards miss| m_qroq

  m_api -->|calls| llm
  m_anthropic -->|calls| llm
  m_online -->|calls| llm
  m_qroq -->|calls| llm

  llm -->|responses| CacheMgr
  CacheMgr -->|returns response| foa
  foa -->|returns result| t_game24
  t_game24 -->|outputs| driver

  driver -->|reads| datasets
  driver -->|writes| figures
  driver -->|writes| examples

  utils -->|helpers for| CacheMgr
  typedefs -->|types for| CacheMgr

  %% Click Events
  click foa "https://github.com/au-clan/cachesaver/blob/main/src/algorithms/foa.py"
  click got "https://github.com/au-clan/cachesaver/blob/main/src/algorithms/got.py"
  click rap "https://github.com/au-clan/cachesaver/blob/main/src/algorithms/rap.py"
  click react "https://github.com/au-clan/cachesaver/blob/main/src/algorithms/react.py"
  click tot_bfs "https://github.com/au-clan/cachesaver/blob/main/src/algorithms/tot_bfs.py"
  click tot_dfs "https://github.com/au-clan/cachesaver/blob/main/src/algorithms/tot_dfs.py"

  click m_anthropic "https://github.com/au-clan/cachesaver/blob/main/src/models/anthropic.py"
  click m_api "https://github.com/au-clan/cachesaver/blob/main/src/models/api.py"
  click m_online "https://github.com/au-clan/cachesaver/blob/main/src/models/online.py"
  click m_qroq "https://github.com/au-clan/cachesaver/blob/main/src/models/qroq.py"

  click t_game24 "https://github.com/au-clan/cachesaver/tree/main/src/tasks/game24/"
  click t_hle "https://github.com/au-clan/cachesaver/tree/main/src/tasks/hle/"
  click t_hotpotqa "https://github.com/au-clan/cachesaver/tree/main/src/tasks/hotpotqa/"
  click t_humaneval "https://github.com/au-clan/cachesaver/tree/main/src/tasks/humaneval/"
  click t_logiqa "https://github.com/au-clan/cachesaver/tree/main/src/tasks/logiqa/"
  click t_matharena "https://github.com/au-clan/cachesaver/tree/main/src/tasks/matharena/"
  click t_scibench "https://github.com/au-clan/cachesaver/tree/main/src/tasks/scibench/"
  click t_sonnet "https://github.com/au-clan/cachesaver/tree/main/src/tasks/sonnetwriting/"

  click utils "https://github.com/au-clan/cachesaver/blob/main/src/utils.py"
  click typedefs "https://github.com/au-clan/cachesaver/blob/main/src/typedefs.py"

  click scripts_abl "https://github.com/au-clan/cachesaver/tree/main/scripts/ablations/"
  click scripts_correctness "https://github.com/au-clan/cachesaver/tree/main/scripts/correctness/"
  click scripts_frameworks "https://github.com/au-clan/cachesaver/tree/main/scripts/frameworks/"
  click scripts_hyper "https://github.com/au-clan/cachesaver/tree/main/scripts/hyper_search/"
  click scripts_motivation "https://github.com/au-clan/cachesaver/tree/main/scripts/motivation/"
  click scripts_namespace "https://github.com/au-clan/cachesaver/tree/main/scripts/namespace/"
  click scripts_old "https://github.com/au-clan/cachesaver/tree/main/scripts/old/"
  click scripts_perms "https://github.com/au-clan/cachesaver/tree/main/scripts/perms/"
  click scripts_single "https://github.com/au-clan/cachesaver/tree/main/scripts/single/"

  click datasets "https://github.com/au-clan/cachesaver/tree/main/datasets/"
  click figures "https://github.com/au-clan/cachesaver/tree/main/figures/"
  click examples "https://github.com/au-clan/cachesaver/tree/main/examples/"

  %% Styles
  classDef internal fill:#cce5ff,stroke:#004085,color:#004085
  classDef drivers fill:#d4edda,stroke:#155724,color:#155724
  classDef external fill:#f8d7da,stroke:#721c24,color:#721c24
  classDef storage fill:#e2e3e5,stroke:#6c757d,color:#6c757d
```
