# anmolwajid3/memory-agent-system architecture

> Architecture diagram of the GitHub repository anmolwajid3/memory-agent-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-06-27). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/anmolwajid3/memory-agent-system
- Repository: https://github.com/anmolwajid3/memory-agent-system

## Overview

1. **What this repository is**
   - This project compares two answer-generation strategies for long-term conversational recall on **LongMemEval-S**:
     - a **full-context baseline** that sends the whole conversation history to the model
     - a **structured memory system** that extracts and retrieves only relevant facts
   - The key architectural boundary is that **each benchmark task is independent**; the system does **not** carry memory across tasks, even though the UI presents them sequentially.

2. **Top-level execution paths**
   - The main user entrypoint is **`gui/app.py`**, which launches a **Streamlit** app for running experiments and viewing results.
   - The benchmark dataset is prepared via **`download_benchmark.py`** and the **`benchmark/`** package.
   - The experiment runtime flows through **`pipeline/runner.py`**, which coordinates agent execution and logging.

3. **Benchmark/data layer**
   - **`benchmark/loader.py`** is responsible for loading the LongMemEval-S tasks and selecting a subset for experiments.
   - This layer is the boundary between external dataset files and the internal agent pipeline.
   - The repo’s README indicates the dataset is auto-downloaded on first run and the default experiment uses a 40-task subset, with a minimum configurable in the UI.

4. **Pipeline orchestration layer**
   - The **`pipeline/`** package is the control plane for each task.
   - **`pipeline/graph.py`** and **`pipeline/state.py`** indicate the system is modeled as a **state machine / LangGraph workflow**.
   - Supporting modules add operational concerns:
     - **`retry.py`** for transient failure handling
     - **`task_timeout.py`** for per-task execution limits
     - **`token_counter.py`** for measuring model cost
     - **`logger.py`** for structured experiment logs
   - This layer is where task input, agent outputs, memory retrieval, evaluation, and final results are stitched together.

5. **Agent subsystem**
   - The **`agents/`** directory contains the main reasoning roles:
     - **`task_agent.py`**: handles the task/question and likely the answer-generation flow
     - **`memory_agent.py`**: manages memory lookup/update actions
     - **`orchestrator.py`**: coordinates agent interactions
     - **`evaluator_agent.py`**: checks answers against expected outcomes / scoring logic
     - **`reflexion_agent.py`**: performs repair or self-correction when the first attempt is insufficient
   - Architecturally, this is the intelligence layer above the pipeline state machine: the pipeline decides *when* agents run, and the agents decide *what* to do.

6. **Memory subsystem**
   - The **`memory/`** package implements the structured-memory condition.
   - **`mem0_client.py`** connects to **Mem0**, which serves as the external memory backend.
   - **`graph_store.py`** suggests memory is organized as a **categorical graph** rather than raw transcript chunks.
   - **`consolidator.py`** likely extracts and merges facts into persistent memory entries.
   - **`retriever.py`** retrieves relevant memory at query time.
   - The main architectural idea here is a two-step memory cycle: **store/consolidate facts after reading history, then retrieve only relevant facts when answering**.

7. **Comparison between the two experimental conditions**
   - The **full-context system** is the simplest path: it forwards the complete conversation history directly into the model.
   - The **memory system** inserts a structured intermediate layer:
     1. read the conversation history
     2. extract salient facts
     3. store them in the graph/Mem0-backed memory
     4. retrieve only query-relevant context
     5. generate the answer using that reduced context
   - This comparison is the core experimental design and is reflected in the repo structure more than in a single monolithic application.

8. **Evaluation and experiment logging**
   - The pipeline writes JSON experiment logs, which are meant to be consumed later rather than only displayed live.
   - The **`analysis/analysis.py`** module turns those logs into plots/summary artifacts.
   - This separates **runtime execution** from **post-hoc analysis**, which is important for reproducible benchmarking.

9. **User interface**
   - **`gui/app.py`** provides a Streamlit dashboard for running the experiment and reviewing outcomes.
   - The UI exposes:
     - experiment execution
     - a logs/results view
     - graph generation after runs complete
   - It is a presentation layer only; the substantive work is still done by the benchmark loader, pipeline, agents, and memory modules.

10. **Configuration and operational dependencies**
   - **`config.py`** and **`.env.example`** centralize environment configuration, especially the **OpenAI API key**.
   - The system targets **Python 3.11+** and uses **OpenAI GPT-4o-mini** as the LLM runtime.
   - The repo also relies on **LangGraph** for orchestration and **Mem0** for memory persistence/retrieval, which materially shape the architecture by splitting reasoning from memory management.

11. **Most important architecture boundaries to capture in a diagram**
   - **UI vs execution engine**: Streamlit triggers runs but does not contain the logic.
   - **Benchmark vs runtime**: dataset loading is separate from task execution.
   - **Orchestration vs agents**: the pipeline controls flow; agents implement behavior.
   - **Raw dialogue vs structured memory**: the memory system transforms conversation into a smaller, queryable representation.
   - **Execution vs analysis**: logs are produced during runs and analyzed afterward.

12. **If you draw the system**
   - Represent the repo as four major zones:
     - **Input/Benchmark**
     - **Orchestration/Pipeline**
     - **Agent + Memory reasoning**
     - **Logging/Analysis/UI**
   - Show two answer paths inside the runtime: **full-context baseline** and **memory-backed retrieval**, both ending in evaluation and logging.

Main entry points: [`pipeline/runner.py`](https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/runner.py) (Task Runner), [`agents/orchestrator.py`](https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/orchestrator.py) (Orchestrator), [`pipeline/graph.py`](https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/graph.py) (Workflow Graph), [`agents/memory_agent.py`](https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/memory_agent.py) (Memory Agent)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_input["Input / Benchmark"]
  node_benchmark_loader[("Benchmark Loader<br/>dataset loader<br/>[loader.py]")]
  node_download_benchmark["Dataset Download<br/>bootstrap script"]
end

subgraph group_runtime["Orchestration / Pipeline"]
  node_runner["Task Runner<br/>experiment runner<br/>[runner.py]"]
  node_workflow_graph{{"Workflow Graph<br/>langgraph flow<br/>[graph.py]"}}
  node_task_state(("Task State<br/>state model<br/>[state.py]"))
  node_logger["Experiment Logs<br/>structured logger<br/>[logger.py]"]
end

subgraph group_reasoning["Agents / Memory"]
  node_orchestrator["Orchestrator<br/>agent coordinator<br/>[orchestrator.py]"]
  node_task_agent["Task Agent<br/>answer agent<br/>[task_agent.py]"]
  node_memory_agent["Memory Agent<br/>memory controller<br/>[memory_agent.py]"]
  node_evaluator_agent["Evaluator<br/>scorer<br/>[evaluator_agent.py]"]
  node_reflexion_agent["Reflexion<br/>repair agent<br/>[reflexion_agent.py]"]
  node_consolidator{{"Consolidator<br/>fact extractor<br/>[consolidator.py]"}}
  node_retriever["Retriever<br/>[retriever.py]"]
  node_graph_store[("Graph Store<br/>memory graph<br/>[graph_store.py]")]
  node_mem0_client[("Mem0 Client<br/>memory backend<br/>[mem0_client.py]")]
end

subgraph group_ops["Logging / Analysis / UI"]
  node_gui_app["Experiment UI<br/>streamlit app<br/>[app.py]"]
  node_analysis["Analysis<br/>[analysis.py]"]
  node_config["Config<br/>[config.py]"]
end

node_gui_app -->|"start run"| node_runner
node_gui_app -->|"view results"| node_analysis
node_download_benchmark -->|"bootstrap"| node_benchmark_loader
node_benchmark_loader -->|"tasks"| node_runner
node_config -->|"settings"| node_gui_app
node_config -->|"llm/api"| node_runner
node_runner -->|"orchestrate"| node_workflow_graph
node_workflow_graph -->|"state"| node_task_state
node_workflow_graph -->|"dispatch"| node_orchestrator
node_orchestrator -->|"answer"| node_task_agent
node_orchestrator -->|"memory"| node_memory_agent
node_orchestrator -->|"score"| node_evaluator_agent
node_task_agent -->|"repair"| node_reflexion_agent
node_memory_agent -->|"store"| node_consolidator
node_consolidator -->|"persist"| node_graph_store
node_graph_store -->|"backend"| node_mem0_client
node_memory_agent -->|"fetch"| node_retriever
node_retriever -->|"query"| node_graph_store
node_evaluator_agent -->|"emit"| node_logger
node_runner -->|"record"| node_logger
node_logger -->|"consume"| node_analysis

click node_gui_app "https://github.com/anmolwajid3/memory-agent-system/blob/main/gui/app.py"
click node_analysis "https://github.com/anmolwajid3/memory-agent-system/blob/main/analysis/analysis.py"
click node_benchmark_loader "https://github.com/anmolwajid3/memory-agent-system/blob/main/benchmark/loader.py"
click node_download_benchmark "https://github.com/anmolwajid3/memory-agent-system/blob/main/download_benchmark.py"
click node_runner "https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/runner.py"
click node_workflow_graph "https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/graph.py"
click node_task_state "https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/state.py"
click node_logger "https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/logger.py"
click node_orchestrator "https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/orchestrator.py"
click node_task_agent "https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/task_agent.py"
click node_memory_agent "https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/memory_agent.py"
click node_evaluator_agent "https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/evaluator_agent.py"
click node_reflexion_agent "https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/reflexion_agent.py"
click node_consolidator "https://github.com/anmolwajid3/memory-agent-system/blob/main/memory/consolidator.py"
click node_retriever "https://github.com/anmolwajid3/memory-agent-system/blob/main/memory/retriever.py"
click node_graph_store "https://github.com/anmolwajid3/memory-agent-system/blob/main/memory/graph_store.py"
click node_mem0_client "https://github.com/anmolwajid3/memory-agent-system/blob/main/memory/mem0_client.py"
click node_config "https://github.com/anmolwajid3/memory-agent-system/blob/main/config.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_benchmark_loader,node_download_benchmark toneBlue
class node_runner,node_workflow_graph,node_task_state,node_logger toneAmber
class node_orchestrator,node_task_agent,node_memory_agent,node_evaluator_agent,node_reflexion_agent,node_consolidator,node_retriever,node_graph_store,node_mem0_client toneMint
class node_gui_app,node_analysis,node_config toneRose
```

## Components

### Input / Benchmark

Dataset acquisition and task loading for LongMemEval-S.

- [Benchmark Loader](https://github.com/anmolwajid3/memory-agent-system/blob/main/benchmark/loader.py): dataset loader · Loads LongMemEval-S tasks and selects experiment subsets. · `benchmark/loader.py`
- [Dataset Download](https://github.com/anmolwajid3/memory-agent-system/blob/main/download_benchmark.py): bootstrap script · Fetches benchmark data on first run. · `download_benchmark.py`

### Orchestration / Pipeline

Execution control plane for each independent task.

- [Task Runner](https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/runner.py): experiment runner · Coordinates each task execution and logging. · `pipeline/runner.py`
- [Workflow Graph](https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/graph.py): langgraph flow · Defines the state-machine workflow for a task. · `pipeline/graph.py`
- [Task State](https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/state.py): state model · Carries task input, intermediate outputs, and results. · `pipeline/state.py`
- [Experiment Logs](https://github.com/anmolwajid3/memory-agent-system/blob/main/pipeline/logger.py): structured logger · Writes JSON logs for later analysis. · `pipeline/logger.py`

### Agents / Memory

Answer generation, memory operations, and recovery logic.

- [Orchestrator](https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/orchestrator.py): agent coordinator · Coordinates the reasoning roles during a task. · `agents/orchestrator.py`
- [Task Agent](https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/task_agent.py): answer agent · Handles question answering and final response generation. · `agents/task_agent.py`
- [Memory Agent](https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/memory_agent.py): memory controller · Manages memory lookup and update actions. · `agents/memory_agent.py`
- [Evaluator](https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/evaluator_agent.py): scorer · Checks answers and produces task evaluation signals. · `agents/evaluator_agent.py`
- [Reflexion](https://github.com/anmolwajid3/memory-agent-system/blob/main/agents/reflexion_agent.py): repair agent · Attempts self-correction after weak answers. · `agents/reflexion_agent.py`
- [Consolidator](https://github.com/anmolwajid3/memory-agent-system/blob/main/memory/consolidator.py): fact extractor · Extracts and merges salient facts into memory. · `memory/consolidator.py`
- [Retriever](https://github.com/anmolwajid3/memory-agent-system/blob/main/memory/retriever.py): context retriever · Fetches query-relevant structured memory. · `memory/retriever.py`
- [Graph Store](https://github.com/anmolwajid3/memory-agent-system/blob/main/memory/graph_store.py): memory graph · Organizes memory as a categorical graph. · `memory/graph_store.py`
- [Mem0 Client](https://github.com/anmolwajid3/memory-agent-system/blob/main/memory/mem0_client.py): memory backend · Connects to external Mem0 persistence. · `memory/mem0_client.py`

### Logging / Analysis / UI

Presentation, logs, and post-hoc analysis.

- [Experiment UI](https://github.com/anmolwajid3/memory-agent-system/blob/main/gui/app.py): streamlit app · Runs experiments and shows results. · `gui/app.py`
- [Analysis](https://github.com/anmolwajid3/memory-agent-system/blob/main/analysis/analysis.py): log analysis · Turns run logs into plots and summaries. · `analysis/analysis.py`
- [Config](https://github.com/anmolwajid3/memory-agent-system/blob/main/config.py): env config · Central environment and API configuration. · `config.py`

## Connections

- Experiment UI → Task Runner: start run · Streamlit triggers experiment execution.
- Experiment UI → Analysis: view results · UI presents post-run summaries and plots.
- Dataset Download → Benchmark Loader: bootstrap · Downloaded data is loaded through the benchmark layer.
- Benchmark Loader → Task Runner: tasks · Provides the selected LongMemEval-S tasks.
- Config → Experiment UI: settings · UI reads environment configuration.
- Config → Task Runner: llm/api · Runtime uses configured model and API credentials.
- Task Runner → Workflow Graph: orchestrate · Runner executes the task workflow.
- Workflow Graph → Task State: state · Workflow advances through task state transitions.
- Workflow Graph → Orchestrator: dispatch · Graph sends control to the agent coordinator.
- Orchestrator → Task Agent: answer · Coordinator invokes the task-solving agent.
- Orchestrator → Memory Agent: memory · Coordinator invokes memory operations when needed.
- Orchestrator → Evaluator: score · Coordinator asks for evaluation of outputs.
- Task Agent → Reflexion: repair · Weak answers can trigger self-correction.
- Memory Agent → Consolidator: store · Converts dialogue into structured facts.
- Consolidator → Graph Store: persist · Writes consolidated facts into graph memory.
- Graph Store → Mem0 Client: backend · Memory graph persists through Mem0.
- Memory Agent → Retriever: fetch · Retrieves relevant memory for the current query.
- Retriever → Graph Store: query · Retrieval reads from the structured memory store.
- Evaluator → Experiment Logs: emit · Evaluation outcomes are written to logs.
- Task Runner → Experiment Logs: record · Runner records execution events and metrics.
- Experiment Logs → Analysis: consume · Analysis reads experiment logs after runs complete.