How juliangeymonat-jpg/mothrag works
1. Overall shape
- MOTHRAG is a Python package for multi-hop question answering.
- The architecture is centered on a deterministic orchestration layer that coordinates retrieval, multiple “reasoning arms,” and arbitration to produce a proof-tree-style answer.
- The top-level user-facing entry point is the package API, with mothrag/core/mothrag.py as the main orchestrator and mothrag/api/simple.py as the simplified public surface.
2. Document ingestion boundary
- Raw sources enter through the loader layer in mothrag/loaders/:
- text, JSON, HTML, PDF, and URL loaders.
- Domain-specific preprocessing lives in mothrag/data/preprocess_wikipedia.py and the Wikipedia plugin in mothrag/plugins/wikipedia.py.
- This layer normalizes heterogeneous inputs into chunks and structured artifacts used by retrieval and graph-building subsystems.
3. Indexing and knowledge representations
- The repo supports both vector retrieval and graph-style retrieval.
- mothrag/graph/ handles graph construction/indexing and OpenIE-related processing.
- mothrag/retrieval/openie.py, mothrag/retrieval/ner.py, and mothrag/core/symbolic_memory.py provide symbolic/structured representations that feed multi-hop reasoning.
- mothrag/core/anchor.py and mothrag/core/decompose.py support anchoring and question decomposition for multi-step retrieval.
4. Embedding and vector storage layer
- Embedding backends are abstracted in mothrag/embedders/ with implementations for OpenAI, Gemini, Cohere, Vertex, and sentence-transformers.
- Vector database adapters live in mothrag/vector_dbs/ and include in-memory, FAISS, Chroma, Pinecone, and Qdrant.
- This layer is a clear infrastructure boundary: the rest of the system depends on a vector-store interface, not a specific backend.
5. Primary retrieval substrate
- Core retrieval logic is in mothrag/core/retrieval/ and mothrag/retrieval/.
- The README’s “bridge retrieval substrate” is implemented by the bridge_haiku family under mothrag/retrieval/bridge_haiku/, which combines ANN retrieval, entity/SVO extraction, fusion, and a judge stage.
- Additional retrieval modes include dense, hybrid graph, infobox, and multimodal retrieval in mothrag/core/retrieval/.
- mothrag/retrieval/chain_filter/ adds post-retrieval filtering based on chain density over graph/triple structure.
6. Reasoning arms
- The system does not rely on one monolithic pipeline; it runs multiple arms in parallel and arbitrates among them.
- Core arms are defined in mothrag/arms/:
- mothgraph_arm.py
- infobox_arm.py
- legacy.py
- base.py
- The README describes a four-arm pool: direct read, decomposition, iterative refinement, and PDD-style duplication of the iterative candidate.
- mothrag/core/arms_runner.py coordinates execution of these arms.
7. Routing and arm applicability
- Query-to-arm selection is handled in mothrag/routing/.
- Important pieces include:
- query_type_classifier.py in mothrag/core/
- arm_applicability.py
- specialist routing for slot-based/adaptive routing
- duplicate-arm handling and infobox routing
- This routing boundary is deterministic and feature-driven, not learned online from dataset identity.
8. Arbitration and abstention
- Final answer selection happens in mothrag/core/arbitrate/.
- Arbitration modules combine evidence from grounding, cross-arm agreement, faithfulness, and abstention handling.
- The system has explicit abstain markers and refusal/abstention logic in mothrag/core/abstain_markers.py and arbitration helpers such as refuse_abstention.py.
- The result is not just an answer string; it is a selected arm, confidence signal, and proof-trace-oriented output.
9. Retry and iterative recovery
- Failed or weakly grounded answers can trigger retry strategies in mothrag/core/retry/strategies/.
- Strategies include query reformulation, sub-question rerouting, active gap querying, soft fallback, bottom-up boost, and iterative extension.
- The iterative_ragnatela/ package contains the multi-step iterative planner/generator path used for repeated retrieval and convergence control.
- This is the main “agentic-style” loop, but it is still orchestrated deterministically by fixed strategy logic.
10. LLM/model service boundaries
- The system is built around commodity API calls rather than local model hosting.
- Readers live in mothrag/readers/ with adapters for OpenAI, Groq, Anthropic, Gemini, and Cohere.
- Embedders and some judges also use external services, especially Gemini and Anthropic in the premium/economy configurations.
- mothrag/core/_api_adapters.py and mothrag/core/api.py provide the abstraction boundary so the rest of the system can swap providers.
11. Graph, routing, and specialized subpipelines
- mothrag/routing/specialist_slot_router.py and specialist_slot_adapters.py support specialist subpipelines.
- mothrag/core/spectral/ and mothrag/core/selective_ensemble.py indicate additional ranking/selection logic used in specialized ensemble or disaggregation flows.
- These subsystems appear to optimize when to invoke a specialist path versus a generic retrieval path.
12. Observability, evaluation, and offline research workflow
- mothrag/eval/ contains the benchmarking and metric pipeline: faithfulness, latency, soft EM, normalization, and iterative evaluation.
- benchmarks/results/ and paper/results/ store experiment outputs and published paper artifacts.
- scripts/route_prospective.py is the main reproduction script referenced by the README.
- tests/ is extensive and acts as a specification for routing, arbitration, retry, graph retrieval, and provider adapters.
13. Operational surface
- mothrag/cli/smoke.py provides a smoke-test CLI path.
- examples/ demonstrates both the quickstart API and custom-corpus flows.
- The repo is documentation-heavy (docs/ and MkDocs config), which suggests the package is intended to be used as a library with reproducible research workflows rather than a standalone service.
14. What to draw as the main data flow
- A useful architecture diagram would show:
- documents entering loaders/preprocessing,
- then embeddings/vector DB and graph/index construction,
- then query intake into routing/classification,
- then multiple arms performing retrieval + reasoning,
- then arbitration and abstention handling,
- then final answer/proof-tree output.
- Retry strategies and iterative ragnatela should be drawn as a feedback loop from weak answers back into retrieval/reformulation.
Main entry points: mothrag.py Mothrag Core · arms_runner.py Arms Runner · arbitrator.py Arbitration · __init__.py Routing
Components 25
Ingestion
Source loading and corpus normalization
- Loaders · Text, JSON, HTML, PDF, and URL inputs
mothrag/loaders/__init__.py - Wikipedia Prep · Wikipedia-specific normalization and plugin integration
mothrag/data/preprocess_wikipedia.py
Representations
Vector, graph, and symbolic knowledge layers
- Vector DBs · Adapters for in-memory, FAISS, Chroma, Pinecone, and Qdrant
mothrag/vector_dbs/base.py - Graph Index · Graph construction and OpenIE-oriented indexing
mothrag/graph/index.py - Symbolic Memory · Structured evidence store used by multi-hop reasoning
mothrag/core/symbolic_memory.py - Dense Retrieval · Dense and multimodal retrieval primitives
mothrag/core/retrieval/dense.py - Hybrid Graph · Graph-aware retrieval over structured evidence
mothrag/core/retrieval/hybrid_graph.py - Bridge Haiku · ANN, entity/SVO extraction, fusion, and judge stage for bridge retrieval
mothrag/retrieval/bridge_haiku/bridge_arm.py - Chain Filter · Filters retrieved chains using graph/triple density
mothrag/retrieval/chain_filter/chain_filter.py
Orchestration
Query routing, multi-arm execution, retries, and arbitration
- Simple API · Simplified entry point for package users
mothrag/api/simple.py - Mothrag Core · Main deterministic coordinator for retrieval, reasoning arms, and final answer assembly
mothrag/core/mothrag.py - Arms Runner · Executes multiple reasoning arms and collects candidate answers
mothrag/core/arms_runner.py - Routing · Query-to-arm selection and specialist dispatch logic
mothrag/routing/__init__.py - Query Type · Classifies question shape to guide routing and retrieval choices
mothrag/core/query_type_classifier.py - Retry Loop · Reformulates or reroutes weak answers back into retrieval
mothrag/core/retry/orchestrator.py - Arbitration · Selects final answer using grounding, agreement, and abstention signals
mothrag/core/arbitrate/arbitrator.py - Abstain Logic · Explicit abstention markers and refusal handling
mothrag/core/abstain_markers.py - Decomposition · Breaks complex questions into sub-questions for multi-hop retrieval
mothrag/core/decompose.py - Anchoring · Links intermediate evidence and supports anchored retrieval
mothrag/core/anchor.py - Iterative Ragnatela · Multi-step repeated retrieval and convergence control
mothrag/iterative_ragnatela/ragnatela_orchestrator.py
Provider Layer
Embedding and LLM API adapters
- Embedders · OpenAI, Gemini, Cohere, Vertex, and sentence-transformer embeddings
mothrag/embedders/__init__.py - Model Reads · OpenAI, Groq, Anthropic, Gemini, and Cohere reader adapters
mothrag/readers/__init__.py
Research & Ops
Evaluation, CLI, examples, and reproduction workflow
- Evaluation · Faithfulness, latency, soft EM, and iterative evaluation
mothrag/eval/pipeline.py - Smoke CLI · Minimal operational entry point for quick validation
mothrag/cli/smoke.py - Examples · Quickstart and custom corpus workflows
examples/00_simple_quickstart.py
Connections 29
- Simple API → Mothrag Core: calls · Public API delegates to core orchestration
- Mothrag Core → Query Type: classifies · Determines query shape before routing
- Mothrag Core → Routing: routes · Selects relevant arm and specialist paths
- Routing → Arms Runner: dispatches · Sends query to chosen reasoning arms
- Query Type → Decomposition: guides · Complex queries trigger decomposition
- Mothrag Core → Decomposition: plans · Creates sub-questions for multi-hop reasoning
- Mothrag Core → Anchoring: anchors · Binds evidence across hops and retrieval steps
- Arms Runner → Dense Retrieval: retrieves · Candidates are sourced from dense/multimodal retrieval
- Arms Runner → Hybrid Graph: retrieves · Graph-aware retrieval supports multi-hop evidence
- Arms Runner → Bridge Haiku: retrieves · Bridge substrate combines ANN and structured extraction
- Arms Runner → Chain Filter: filters · Post-retrieval chain gating improves precision
- Dense Retrieval → Vector DBs: uses · Vector backend provides ANN storage and search
- Bridge Haiku → Embedders: embeds · Bridge retrieval depends on embedding providers
- Loaders → Embedders: feeds · Loaded chunks are embedded for indexing and retrieval
- Loaders → Graph Index: normalizes · Raw sources become graph/index artifacts
- Wikipedia Prep → Loaders: preprocesses · Wikipedia-specific normalization enters the loader boundary
- Graph Index → Symbolic Memory: stores · Structured graph facts populate symbolic memory
- Symbolic Memory → Hybrid Graph: serves · Structured memory supports graph-aware retrieval
- Routing → Retry Loop: may reroute · Arm selection can trigger recovery strategies
- Retry Loop → Iterative Ragnatela: extends · Iterative loop performs repeated retrieval and convergence control
- Iterative Ragnatela → Arms Runner: retries · Weak answers can re-enter arm execution
- Arms Runner → Arbitration: candidates · Arm outputs are scored and compared
- Arbitration → Abstain Logic: checks · Arbitration considers refusal and abstention signals
- Arbitration → Mothrag Core: returns · Selected answer and trace flow back to core output
- Model Reads → Arbitration: supports · Judging and verification can use model providers
- Model Reads → Bridge Haiku: supports · Bridge stages may call reader providers for extraction/judging
- Evaluation → Mothrag Core: measures · Benchmarking exercises orchestration and retrieval quality
- Smoke CLI → Mothrag Core: invokes · Operational smoke path drives the package core
- Examples → Simple API: demonstrates · Examples show the public entry surface