# yousha806/graph-webagents architecture

> Architecture diagram of the GitHub repository yousha806/graph-webagents, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/yousha806/graph-webagents
- Repository: https://github.com/yousha806/graph-webagents

## Overview

Graph-WebAgents is a Multimodal Mind2Web data-preparation and web-agent research repository. Its documented workflow is to obtain dataset splits, optionally enrich them with accessibility-tree representations, load batches, run baseline inference, and evaluate predictions. The graph preserves the major model families and evaluation tools. Some inference scripts are templates or baselines; connections between unsampled modules are included only where documented, and are otherwise omitted.

Main entry points: [`mind2web_dataloader.py`](https://github.com/yousha806/graph-webagents/blob/main/mind2web_dataloader.py) (Mind2Web DataLoader), [`inference/batch_inference.py`](https://github.com/yousha806/graph-webagents/blob/main/inference/batch_inference.py) (Batch Inference Template), [`precompute_axtree.py`](https://github.com/yousha806/graph-webagents/blob/main/precompute_axtree.py) (AXTree Precompute), [`baselines/models/vlm_inference.py`](https://github.com/yousha806/graph-webagents/blob/main/baselines/models/vlm_inference.py) (VLM Baseline)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_preparation["Dataset Preparation"]
  node_local_splits[("Local Splits")]
  node_download["Split Downloader"]
  node_axtree["AXTree Precompute"]
  node_axtree_splits[("Enriched Splits")]
end

subgraph group_access["Dataset Access"]
  node_loader["Mind2Web DataLoader"]
end

subgraph group_models["Agent Baselines"]
  node_gat_pretrain["GAT Pretraining<br/>[GAT_Pretrain.py]"]
  node_gnn["Multimodal GNN<br/>[multimodal_GNN.py]"]
  node_gnn_cross["Cross-Attention GNN"]
  node_mmml["Graph LoRA Model"]
  node_instruct["Multimodal Instruct"]
  node_qwen_instruct["Qwen Instruct"]
  node_text_only["Text-Only Baseline<br/>[text_only.py]"]
  node_vlm["VLM Baseline<br/>[vlm_inference.py]"]
end

subgraph group_evaluation["Inference Evaluation"]
  node_batch["Batch Inference Template<br/>[batch_inference.py]"]
  node_eval_next["Next-Action Evaluator"]
  node_vlm_eval["Prediction Metrics"]
end

node_user(("Researcher"))
node_hf_dataset[("Mind2Web Dataset")]
node_chromium["Headless Chromium"]

node_user -->|"selects splits"| node_download
node_download -->|"downloads"| node_hf_dataset
node_download -->|"saves splits"| node_local_splits
node_user -->|"requests enrichment"| node_axtree
node_axtree -->|"loads split"| node_hf_dataset
node_axtree -->|"renders HTML"| node_chromium
node_axtree -->|"saves enriched data"| node_axtree_splits
node_user -->|"requests batches"| node_loader
node_local_splits -->|"loads local split"| node_loader
node_hf_dataset -->|"loads directly"| node_loader
node_loader -.->|"provides dataset inputs"| node_gat_pretrain
node_loader -.->|"provides dataset inputs"| node_gnn
node_loader -.->|"provides dataset inputs"| node_gnn_cross
node_loader -.->|"provides dataset inputs"| node_mmml
node_loader -.->|"provides dataset inputs"| node_instruct
node_loader -.->|"provides dataset inputs"| node_qwen_instruct
node_loader -.->|"provides dataset inputs"| node_text_only
node_loader -.->|"provides dataset inputs"| node_vlm
node_user -->|"runs inference"| node_batch
node_batch -.->|"dispatches baseline"| node_vlm
node_batch -.->|"dispatches baseline"| node_gnn
node_batch -.->|"dispatches baseline"| node_gnn_cross
node_batch -.->|"dispatches baseline"| node_text_only
node_vlm -.->|"emits predictions"| node_eval_next
node_vlm -.->|"emits predictions"| node_vlm_eval
node_user -->|"evaluates predictions"| node_eval_next
node_user -->|"evaluates predictions"| node_vlm_eval

click node_download "https://github.com/yousha806/graph-webagents/blob/main/download_mind2web.py"
click node_axtree "https://github.com/yousha806/graph-webagents/blob/main/precompute_axtree.py"
click node_loader "https://github.com/yousha806/graph-webagents/blob/main/mind2web_dataloader.py"
click node_gat_pretrain "https://github.com/yousha806/graph-webagents/blob/main/baselines/models/GAT_Pretrain.py"
click node_gnn "https://github.com/yousha806/graph-webagents/blob/main/baselines/models/multimodal_GNN.py"
click node_gnn_cross "https://github.com/yousha806/graph-webagents/blob/main/baselines/models/multimodal_GNN_Cross.py"
click node_mmml "https://github.com/yousha806/graph-webagents/blob/main/baselines/models/mmml_metrics_merged.py"
click node_instruct "https://github.com/yousha806/graph-webagents/blob/main/baselines/models/multimodal_instruct.py"
click node_qwen_instruct "https://github.com/yousha806/graph-webagents/blob/main/baselines/models/multimodal_qwen_instruct.py"
click node_text_only "https://github.com/yousha806/graph-webagents/blob/main/baselines/models/text_only.py"
click node_vlm "https://github.com/yousha806/graph-webagents/blob/main/baselines/models/vlm_inference.py"
click node_batch "https://github.com/yousha806/graph-webagents/blob/main/inference/batch_inference.py"
click node_eval_next "https://github.com/yousha806/graph-webagents/blob/main/eval_next_action.py"
click node_vlm_eval "https://github.com/yousha806/graph-webagents/blob/main/inference/vlm_inference_script.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_local_splits,node_download,node_axtree,node_axtree_splits toneBlue
class node_loader,node_hf_dataset toneAmber
class node_gat_pretrain,node_gnn,node_gnn_cross,node_mmml,node_instruct,node_qwen_instruct,node_text_only,node_vlm toneMint
class node_batch,node_eval_next,node_vlm_eval toneRose
class node_user,node_chromium toneIndigo
```

## Components

### Dataset Preparation

- **Local Splits**
- [Split Downloader](https://github.com/yousha806/graph-webagents/blob/main/download_mind2web.py): `download_mind2web.py`
- [AXTree Precompute](https://github.com/yousha806/graph-webagents/blob/main/precompute_axtree.py): `precompute_axtree.py`
- **Enriched Splits**

### Dataset Access

- [Mind2Web DataLoader](https://github.com/yousha806/graph-webagents/blob/main/mind2web_dataloader.py): `mind2web_dataloader.py`

### Agent Baselines

- [GAT Pretraining](https://github.com/yousha806/graph-webagents/blob/main/baselines/models/GAT_Pretrain.py): `baselines/models/GAT_Pretrain.py`
- [Multimodal GNN](https://github.com/yousha806/graph-webagents/blob/main/baselines/models/multimodal_GNN.py): `baselines/models/multimodal_GNN.py`
- [Cross-Attention GNN](https://github.com/yousha806/graph-webagents/blob/main/baselines/models/multimodal_GNN_Cross.py): `baselines/models/multimodal_GNN_Cross.py`
- [Graph LoRA Model](https://github.com/yousha806/graph-webagents/blob/main/baselines/models/mmml_metrics_merged.py): `baselines/models/mmml_metrics_merged.py`
- [Multimodal Instruct](https://github.com/yousha806/graph-webagents/blob/main/baselines/models/multimodal_instruct.py): `baselines/models/multimodal_instruct.py`
- [Qwen Instruct](https://github.com/yousha806/graph-webagents/blob/main/baselines/models/multimodal_qwen_instruct.py): `baselines/models/multimodal_qwen_instruct.py`
- [Text-Only Baseline](https://github.com/yousha806/graph-webagents/blob/main/baselines/models/text_only.py): `baselines/models/text_only.py`
- [VLM Baseline](https://github.com/yousha806/graph-webagents/blob/main/baselines/models/vlm_inference.py): `baselines/models/vlm_inference.py`

### Inference Evaluation

- [Batch Inference Template](https://github.com/yousha806/graph-webagents/blob/main/inference/batch_inference.py): `inference/batch_inference.py`
- [Next-Action Evaluator](https://github.com/yousha806/graph-webagents/blob/main/eval_next_action.py): `eval_next_action.py`
- [Prediction Metrics](https://github.com/yousha806/graph-webagents/blob/main/inference/vlm_inference_script.py): `inference/vlm_inference_script.py`

### Other components

- **Researcher**
- **Mind2Web Dataset**
- **Headless Chromium**

## Connections

- Researcher → Split Downloader: selects splits (evidence: `README.md`)
- Split Downloader → Mind2Web Dataset: downloads (evidence: `download_mind2web.py`)
- Split Downloader → Local Splits: saves splits (evidence: `download_mind2web.py`)
- Researcher → AXTree Precompute: requests enrichment (evidence: `README.md`)
- AXTree Precompute → Mind2Web Dataset: loads split (evidence: `precompute_axtree.py`)
- AXTree Precompute → Headless Chromium: renders HTML (evidence: `precompute_axtree.py`)
- AXTree Precompute → Enriched Splits: saves enriched data (evidence: `precompute_axtree.py`)
- Researcher → Mind2Web DataLoader: requests batches (evidence: `README.md`)
- Local Splits → Mind2Web DataLoader: loads local split (evidence: `mind2web_dataloader.py`)
- Mind2Web Dataset → Mind2Web DataLoader: loads directly (evidence: `README.md`)
- Mind2Web DataLoader → GAT Pretraining: provides dataset inputs
- Mind2Web DataLoader → Multimodal GNN: provides dataset inputs
- Mind2Web DataLoader → Cross-Attention GNN: provides dataset inputs
- Mind2Web DataLoader → Graph LoRA Model: provides dataset inputs
- Mind2Web DataLoader → Multimodal Instruct: provides dataset inputs
- Mind2Web DataLoader → Qwen Instruct: provides dataset inputs
- Mind2Web DataLoader → Text-Only Baseline: provides dataset inputs
- Mind2Web DataLoader → VLM Baseline: provides dataset inputs
- Researcher → Batch Inference Template: runs inference (evidence: `inference/batch_inference.py`)
- Batch Inference Template → VLM Baseline: dispatches baseline
- Batch Inference Template → Multimodal GNN: dispatches baseline
- Batch Inference Template → Cross-Attention GNN: dispatches baseline
- Batch Inference Template → Text-Only Baseline: dispatches baseline
- VLM Baseline → Next-Action Evaluator: emits predictions
- VLM Baseline → Prediction Metrics: emits predictions
- Researcher → Next-Action Evaluator: evaluates predictions (evidence: `eval_next_action.py`)
- Researcher → Prediction Metrics: evaluates predictions (evidence: `inference/vlm_inference_script.py`)