How lmarena/search-arena works
Search Arena is a research-reproduction repository for analyzing search-augmented LLM conversations, user preferences, intent, and citation grounding. Researchers start with the published battle datasets or processed comparison sets, then use annotation scripts and analysis notebooks to produce classifications, grounding labels, statistical comparisons, and visual results. The graph preserves the documented analysis branches as well as the excerpted annotation stages. Notebook internals were not sampled, so their relationships are based on README descriptions rather than inspected implementation.
Main entry points: search_arena_24k.jsonl Search battles · intent_classification/annotation.py Intent annotation · citation_attribution_analysis/annotation.py Citation attribution · preference_analysis.ipynb Preference analysis
Components 17
Research data
- Search battles
data/search_arena_24k.jsonl - Text battles
data/text_arena_battles.jsonl - Difference sets
data/describing_diffs - Intent labels
- Citation inputs
- Grounding results
data/citation_attribution_data.jsonl
Annotation workflows
- Intent annotation
scripts/intent_classification/annotation.py - Intent prompts
scripts/intent_classification/utils.py - Citation attribution
scripts/citation_attribution_analysis/annotation.py - Citation rules
scripts/citation_attribution_analysis/utils.py
Analysis workflows
- Prompt exploration
scripts/data_eda.ipynb - Preference analysis
scripts/preference_analysis.ipynb - Cross-setting analysis
scripts/cross_setting_analysis.ipynb - Battle statistics
scripts/utils.py - Citation analysis
scripts/citation_attribution_analysis - Intent analysis
scripts/intent_classification
Other components
- Researcher
Connections 19
- Researcher → Intent annotation: runs annotation
README.md - Search battles → Intent annotation: provides prompts
scripts/intent_classification/annotation.py - Intent annotation → Intent prompts: builds requests
scripts/intent_classification/annotation.py - Intent annotation → Intent labels: produces labels
README.md - Researcher → Citation attribution: runs evaluation
README.md - Citation inputs → Citation attribution: provides traces and content
scripts/citation_attribution_analysis/annotation.py - Citation attribution → Citation rules: uses instructions
scripts/citation_attribution_analysis/annotation.py - Citation attribution → Grounding results: writes results
scripts/citation_attribution_analysis/annotation.py - Search battles → Prompt exploration: supplies prompts
README.md - Researcher → Prompt exploration: explores
README.md - Search battles → Preference analysis: supplies battles
README.md - Researcher → Preference analysis: analyzes preferences
README.md - Preference analysis → Battle statistics: uses statistics
- Search battles → Cross-setting analysis: supports comparisons
README.md - Text battles → Cross-setting analysis: supports comparisons
README.md - Researcher → Cross-setting analysis: compares settings
README.md - Difference sets → Researcher: enables comparisons
README.md - Grounding results → Citation analysis: supports analysis
README.md - Intent labels → Intent analysis: supports analysis