How mukeshram-07/spanchor works
SPANCHOR is a local-first library and CLI for regression-testing RAG retrieval against source-document spans rather than chunk IDs. Users prepare canonical documents and gold anchors, adapt retrieval output or provide spans, evaluate per-query evidence coverage, then compare baseline and candidate runs under thresholds and inspect reports or gate CI. The graph preserves annotation, validation, serialization, and framework adapters as supporting capabilities. Some subsystem links are based on README workflow statements because their implementation files were not excerpted.
Main entry points: cli.py CLI commands · engine.py Evaluation engine · compare.py Regression comparison · document.py Source documents
Components 22
Source and gold data
- Canonical text
src/spanchor/canonical/normalize.py - Source documents
src/spanchor/models/document.py - Gold anchors
src/spanchor/models/anchor.py - Anchor annotation
src/spanchor/annotation/add.py - Locate evidence
src/spanchor/annotation/locate.py - Corpus validation
src/spanchor/validation.py - Gold queries
src/spanchor/models/query.py
Retrieval inputs
- Result adapters
src/spanchor/adapters/generic.py - Retrieval results
src/spanchor/models/retrieval.py - Chunk-to-span mapping
src/spanchor/mapping/mapper.py - Optional adapters
src/spanchor/adapters/langchain.py
Evaluation and regression
- Evaluation engine
src/spanchor/evaluation/engine.py - Span metrics
src/spanchor/evaluation/intervals.py - Evaluation run
src/spanchor/models/run.py - Regression comparison
src/spanchor/comparison/compare.py - Metric policy
src/spanchor/comparison/policy.py
Interfaces and persistence
- CLI commands
src/spanchor/cli.py - Run reports
src/spanchor/reporting/markdown.py - Run JSON storage
src/spanchor/storage/json_io.py - Gold JSONL storage
src/spanchor/storage/jsonl.py
Other components
- Benchmark author
- RAG retriever
Connections 25
- Benchmark author → CLI commands: runs commands
README.md - Benchmark author → Anchor annotation: creates anchors
README.md - Canonical text → Source documents: normalizes text
src/spanchor/models/document.py - Source documents → Gold anchors: anchors spans
README.md - Anchor annotation → Locate evidence: locates evidence
src/spanchor/annotation/add.py - CLI commands → Corpus validation: validates corpus
src/spanchor/cli.py - CLI commands → Locate evidence: locates text
src/spanchor/cli.py - Benchmark author → RAG retriever: queries RAG
README.md - RAG retriever → Result adapters: returns chunks
README.md - Optional adapters → Retrieval results: converts results
src/spanchor/adapters/langchain.py - Result adapters → Retrieval results: converts results
src/spanchor/adapters/generic.py - CLI commands → Gold JSONL storage: loads gold data
src/spanchor/cli.py - Retrieval results → Evaluation engine: supplies results
README.md - Gold queries → Evaluation engine: supplies gold
README.md - Source documents → Evaluation engine: supplies corpus
README.md - Evaluation engine → Chunk-to-span mapping: maps chunks
src/spanchor/evaluation/engine.py - Evaluation engine → Span metrics: computes metrics
src/spanchor/evaluation/engine.py - Evaluation engine → Evaluation run: returns run
README.md - Evaluation run → Regression comparison: compares runs
README.md - Metric policy → Regression comparison: sets thresholds
README.md - Regression comparison → Run reports: feeds report
src/spanchor/reporting/markdown.py - Evaluation run → Run JSON storage: persists run
src/spanchor/storage/json_io.py - CLI commands → Evaluation engine: runs evaluation
src/spanchor/cli.py - CLI commands → Regression comparison: runs comparison
src/spanchor/cli.py - CLI commands → Run reports: writes report
src/spanchor/cli.py