# kimdanny/matm architecture

> Architecture diagram of the GitHub repository kimdanny/matm, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/kimdanny/matm
- Repository: https://github.com/kimdanny/matm

## Overview

MATM lets producer agents contribute action-observation trajectories to a shared index, then lets consumer agents retrieve relevant segments during task execution. The runtime selects an environment and retrieval policy, adds retrieved context to agent decisions, and records outcomes; online memory can add runtime trajectories back to the repository. The README also documents an LTR reranking and training-data pipeline. Edges reflect README statements and SOURCE INDEX references; unsampled implementation details are not assumed.

Main entry points: [`traj_retrieval/run_evaluation.py`](https://github.com/kimdanny/matm/blob/main/traj_retrieval/run_evaluation.py) (ALFWorld runner), [`traj_retrieval/run_webarena.py`](https://github.com/kimdanny/matm/blob/main/traj_retrieval/run_webarena.py) (WebArena runner), [`traj_retrieval/core/retrieval_strategy.py`](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/retrieval_strategy.py) (Retrieval manager), [`traj_retrieval/preprocess`](https://github.com/kimdanny/matm/tree/main/traj_retrieval/preprocess) (Trajectory staging and indexing)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_runtime["Agent runtime"]
  node_alf_run["ALFWorld runner<br/>[run_evaluation.py]"]
  node_web_run["WebArena runner<br/>[run_webarena.py]"]
  node_async_loop["Async episode loop"]
  node_sync_loop["Sync episode loop"]
  node_async_policy["Async action policies"]
  node_sync_policy["Sync action policies<br/>[sync_strategies.py]"]
  node_retrieval_handler["Async retrieval handler"]
  node_sync_retrieval_handler["Sync retrieval handler"]
  node_retrieval_manager["Retrieval manager"]
end

subgraph group_memory["Trajectory memory"]
  node_lancedb_retrieval["Trajectory search"]
  node_lancedb[("LanceDB index<br/>[lancedb_client.py]")]
  node_alf_memory["ALFWorld online memory"]
  node_web_memory["WebArena online memory"]
  node_trajectory_staging["Trajectory staging and indexing"]
end

subgraph group_environments["Task environments"]
  node_env_factory["Environment factory"]
  node_env_contract["Environment interface<br/>[base_handler.py]"]
  node_alf_handler["ALFWorld handler"]
  node_web_handler["WebArena handler"]
end

subgraph group_learning["Ranking and evaluation"]
  node_ltr_features["LTR feature extraction"]
  node_ltr_rankers["LTR rankers<br/>[base.py]"]
  node_ltr_data["LTR training data"]
  node_evaluation["Evaluation and metrics"]
  node_reranker["Reranker orchestration"]
end

node_producer(("Producer agents"))
node_consumer(("Consumer agents"))

node_producer -->|"contribute trajectories"| node_trajectory_staging
node_trajectory_staging -->|"build shared index"| node_lancedb
node_consumer -->|"start evaluation"| node_alf_run
node_alf_run -->|"runs episodes"| node_async_loop
node_alf_run -->|"creates environments"| node_env_factory
node_alf_run -->|"configures retrieval"| node_retrieval_manager
node_alf_run -->|"commits trajectories"| node_alf_memory
node_alf_run -->|"collects metrics"| node_evaluation
node_consumer -->|"start evaluation"| node_web_run
node_web_run -->|"runs episodes"| node_sync_loop
node_web_run -->|"creates environments"| node_env_factory
node_web_run -->|"configures retrieval"| node_retrieval_manager
node_web_run -->|"commits trajectories"| node_web_memory
node_web_run -->|"collects metrics"| node_evaluation
node_async_loop -->|"uses retrieval"| node_retrieval_handler
node_async_loop -->|"selects policy"| node_async_policy
node_sync_loop -->|"uses retrieval"| node_sync_retrieval_handler
node_sync_loop -->|"selects policy"| node_sync_policy
node_env_factory -->|"implements interface"| node_env_contract
node_alf_handler -.->|"implements interface"| node_env_contract
node_web_handler -.->|"implements interface"| node_env_contract
node_retrieval_handler -->|"requests retrieval"| node_retrieval_manager
node_sync_retrieval_handler -->|"requests retrieval"| node_retrieval_manager
node_retrieval_manager -->|"selects search"| node_lancedb_retrieval
node_lancedb_retrieval -.->|"queries index"| node_lancedb
node_alf_memory -->|"formats trajectories"| node_trajectory_staging
node_web_memory -->|"formats trajectories"| node_trajectory_staging
node_ltr_data -.->|"prepares ranking features"| node_ltr_features
node_ltr_features -.->|"scores candidates"| node_ltr_rankers
node_retrieval_manager -.->|"reranks results"| node_reranker
node_reranker -.->|"uses trained rankers"| node_ltr_rankers
node_ltr_data -.->|"trains rankers"| node_ltr_rankers
node_evaluation -.->|"supports training data"| node_ltr_data

click node_alf_run "https://github.com/kimdanny/matm/blob/main/traj_retrieval/run_evaluation.py"
click node_web_run "https://github.com/kimdanny/matm/blob/main/traj_retrieval/run_webarena.py"
click node_async_loop "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/async_with_traj_rag.py"
click node_sync_loop "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/sync_with_traj_rag.py"
click node_async_policy "https://github.com/kimdanny/matm/blob/main/traj_retrieval/strategies/async_strategies.py"
click node_sync_policy "https://github.com/kimdanny/matm/blob/main/traj_retrieval/strategies/sync_strategies.py"
click node_retrieval_handler "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/retrieval_handler.py"
click node_sync_retrieval_handler "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/retrieval_handler_sync.py"
click node_retrieval_manager "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/retrieval_strategy.py"
click node_env_factory "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/environment_factory.py"
click node_env_contract "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/base_handler.py"
click node_alf_handler "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/alfworld_handler.py"
click node_web_handler "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/webarena_handler.py"
click node_lancedb_retrieval "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/lancedb_retrieval.py"
click node_lancedb "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/lancedb_client.py"
click node_alf_memory "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/alfworld_online_memory.py"
click node_web_memory "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/webarena_online_memory.py"
click node_trajectory_staging "https://github.com/kimdanny/matm/tree/main/traj_retrieval/preprocess"
click node_ltr_features "https://github.com/kimdanny/matm/blob/main/ltr/runtime_features.py"
click node_ltr_rankers "https://github.com/kimdanny/matm/blob/main/ltr/models/base.py"
click node_ltr_data "https://github.com/kimdanny/matm/tree/main/traj_retrieval/evaluation/ltr_data"
click node_evaluation "https://github.com/kimdanny/matm/blob/main/traj_retrieval/evaluation/evaluate_trajectories.py"
click node_reranker "https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/reranker_orchestrator.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_alf_run,node_web_run,node_async_loop,node_sync_loop,node_async_policy,node_sync_policy,node_retrieval_handler,node_sync_retrieval_handler,node_retrieval_manager toneBlue
class node_lancedb_retrieval,node_lancedb,node_alf_memory,node_web_memory,node_trajectory_staging toneAmber
class node_env_factory,node_env_contract,node_alf_handler,node_web_handler toneMint
class node_ltr_features,node_ltr_rankers,node_ltr_data,node_evaluation,node_reranker toneRose
class node_producer,node_consumer toneIndigo
```

## Components

### Agent runtime

- [ALFWorld runner](https://github.com/kimdanny/matm/blob/main/traj_retrieval/run_evaluation.py): `traj_retrieval/run_evaluation.py`
- [WebArena runner](https://github.com/kimdanny/matm/blob/main/traj_retrieval/run_webarena.py): `traj_retrieval/run_webarena.py`
- [Async episode loop](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/async_with_traj_rag.py): `traj_retrieval/core/async_with_traj_rag.py`
- [Sync episode loop](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/sync_with_traj_rag.py): `traj_retrieval/core/sync_with_traj_rag.py`
- [Async action policies](https://github.com/kimdanny/matm/blob/main/traj_retrieval/strategies/async_strategies.py): `traj_retrieval/strategies/async_strategies.py`
- [Sync action policies](https://github.com/kimdanny/matm/blob/main/traj_retrieval/strategies/sync_strategies.py): `traj_retrieval/strategies/sync_strategies.py`
- [Async retrieval handler](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/retrieval_handler.py): `traj_retrieval/core/retrieval_handler.py`
- [Sync retrieval handler](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/retrieval_handler_sync.py): `traj_retrieval/core/retrieval_handler_sync.py`
- [Retrieval manager](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/retrieval_strategy.py): `traj_retrieval/core/retrieval_strategy.py`

### Trajectory memory

- [Trajectory search](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/lancedb_retrieval.py): `traj_retrieval/core/lancedb_retrieval.py`
- [LanceDB index](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/lancedb_client.py): `traj_retrieval/core/lancedb_client.py`
- [ALFWorld online memory](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/alfworld_online_memory.py): `traj_retrieval/core/alfworld_online_memory.py`
- [WebArena online memory](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/webarena_online_memory.py): `traj_retrieval/core/webarena_online_memory.py`
- [Trajectory staging and indexing](https://github.com/kimdanny/matm/tree/main/traj_retrieval/preprocess): `traj_retrieval/preprocess`

### Task environments

- [Environment factory](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/environment_factory.py): `traj_retrieval/core/environment_factory.py`
- [Environment interface](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/base_handler.py): `traj_retrieval/core/base_handler.py`
- [ALFWorld handler](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/alfworld_handler.py): `traj_retrieval/core/alfworld_handler.py`
- [WebArena handler](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/webarena_handler.py): `traj_retrieval/core/webarena_handler.py`

### Ranking and evaluation

- [LTR feature extraction](https://github.com/kimdanny/matm/blob/main/ltr/runtime_features.py): `ltr/runtime_features.py`
- [LTR rankers](https://github.com/kimdanny/matm/blob/main/ltr/models/base.py): `ltr/models/base.py`
- [LTR training data](https://github.com/kimdanny/matm/tree/main/traj_retrieval/evaluation/ltr_data): `traj_retrieval/evaluation/ltr_data`
- [Evaluation and metrics](https://github.com/kimdanny/matm/blob/main/traj_retrieval/evaluation/evaluate_trajectories.py): `traj_retrieval/evaluation/evaluate_trajectories.py`
- [Reranker orchestration](https://github.com/kimdanny/matm/blob/main/traj_retrieval/core/reranker_orchestrator.py): `traj_retrieval/core/reranker_orchestrator.py`

### Other components

- **Producer agents**
- **Consumer agents**

## Connections

- Producer agents → Trajectory staging and indexing: contribute trajectories (evidence: `README.md`)
- Trajectory staging and indexing → LanceDB index: build shared index (evidence: `README.md`)
- Consumer agents → ALFWorld runner: start evaluation (evidence: `README.md`)
- ALFWorld runner → Async episode loop: runs episodes (evidence: `traj_retrieval/run_evaluation.py`)
- ALFWorld runner → Environment factory: creates environments (evidence: `traj_retrieval/run_evaluation.py`)
- ALFWorld runner → Retrieval manager: configures retrieval (evidence: `traj_retrieval/run_evaluation.py`)
- ALFWorld runner → ALFWorld online memory: commits trajectories (evidence: `traj_retrieval/run_evaluation.py`)
- ALFWorld runner → Evaluation and metrics: collects metrics (evidence: `traj_retrieval/run_evaluation.py`)
- Consumer agents → WebArena runner: start evaluation (evidence: `README.md`)
- WebArena runner → Sync episode loop: runs episodes (evidence: `traj_retrieval/run_webarena.py`)
- WebArena runner → Environment factory: creates environments (evidence: `traj_retrieval/run_webarena.py`)
- WebArena runner → Retrieval manager: configures retrieval (evidence: `traj_retrieval/run_webarena.py`)
- WebArena runner → WebArena online memory: commits trajectories (evidence: `traj_retrieval/run_webarena.py`)
- WebArena runner → Evaluation and metrics: collects metrics (evidence: `traj_retrieval/run_webarena.py`)
- Async episode loop → Async retrieval handler: uses retrieval (evidence: `traj_retrieval/core/async_with_traj_rag.py`)
- Async episode loop → Async action policies: selects policy (evidence: `traj_retrieval/core/async_with_traj_rag.py`)
- Sync episode loop → Sync retrieval handler: uses retrieval (evidence: `traj_retrieval/core/sync_with_traj_rag.py`)
- Sync episode loop → Sync action policies: selects policy (evidence: `traj_retrieval/core/sync_with_traj_rag.py`)
- Environment factory → Environment interface: implements interface (evidence: `traj_retrieval/core/environment_factory.py`)
- ALFWorld handler → Environment interface: implements interface (evidence: `traj_retrieval/core/alfworld_handler.py`)
- WebArena handler → Environment interface: implements interface (evidence: `traj_retrieval/core/webarena_handler.py`)
- Async retrieval handler → Retrieval manager: requests retrieval (evidence: `traj_retrieval/core/retrieval_handler.py`)
- Sync retrieval handler → Retrieval manager: requests retrieval (evidence: `traj_retrieval/core/retrieval_handler_sync.py`)
- Retrieval manager → Trajectory search: selects search (evidence: `traj_retrieval/core/retrieval_strategy.py`)
- Trajectory search → LanceDB index: queries index
- ALFWorld online memory → Trajectory staging and indexing: formats trajectories (evidence: `traj_retrieval/core/alfworld_online_memory.py`)
- WebArena online memory → Trajectory staging and indexing: formats trajectories (evidence: `traj_retrieval/core/webarena_online_memory.py`)
- LTR training data → LTR feature extraction: prepares ranking features
- LTR feature extraction → LTR rankers: scores candidates (evidence: `README.md`)
- Retrieval manager → Reranker orchestration: reranks results (evidence: `README.md`)
- Reranker orchestration → LTR rankers: uses trained rankers (evidence: `README.md`)
- LTR training data → LTR rankers: trains rankers (evidence: `README.md`)
- Evaluation and metrics → LTR training data: supports training data (evidence: `README.md`)
