How halasadoumi/scientific-agent-that-corrects-itself works
Scientific Verification Agent accepts session questions and files, extracts and chunks document evidence, retrieves and ranks passages, drafts answers, performs a separate claim-check retrieval and model call, and records results for audit. Uploads may be processed synchronously or through Redis-backed jobs; arXiv lookup and semantic embeddings are optional branches. The API owns orchestration, while catalog and object storage provide persistence. The map follows README-described flows where excerpts do not expose route internals; unsampled implementation details are not inferred.
Main entry points: main.py Session API · jobs.py Ingestion job · catalog.py Evidence catalog · pipeline.py Document extraction
Components 18
Ingestion
- Input schemas
src/agent/schemas/ingestion.py - Document extraction
src/agent/ingestion/pipeline.py - Ingestion job
src/agent/ingestion/jobs.py
Evidence retrieval
- Query planning
src/agent/retrieval/planning.py - Search cache
src/agent/retrieval/cache.py - Semantic embeddings
src/agent/retrieval/embeddings.py - arXiv search
src/agent/retrieval/scholar.py
Answer verification
- Answer and claim checks
src/agent/generation.py
API and jobs
- Session API
src/agent/api/main.py - Background worker
src/agent/ingestion/worker.py - Runtime settings
src/agent/config.py
Persistence
- Evidence catalog
src/agent/storage/catalog.py - Original file store
src/agent/storage/objects.py
Other components
- Researcher
- Browser test bench
src/agent/api/static/index.html - Redis Streams
- Embedding endpoint
- Model endpoint
Connections 25
- Researcher → Browser test bench: uses
README.md - Browser test bench → Session API: uploads through
README.md - Researcher → Session API: submits inputs
README.md - Session API → Input schemas: validates inputs
src/agent/api/main.py - Session API → Original file store: stores originals
README.md - Session API → Document extraction: processes uploads
README.md - Document extraction → Evidence catalog: indexes chunks
src/agent/ingestion/jobs.py - Session API → Redis Streams: enqueues jobs
README.md - Background worker → Redis Streams: consumes jobs
README.md - Background worker → Ingestion job: runs
src/agent/ingestion/worker.py - Ingestion job → Original file store: loads originals
src/agent/ingestion/jobs.py - Ingestion job → Document extraction: parses and chunks
src/agent/ingestion/jobs.py - Ingestion job → Semantic embeddings: embeds chunks
src/agent/ingestion/jobs.py - Ingestion job → Evidence catalog: writes documents
src/agent/ingestion/jobs.py - Session API → Evidence catalog: searches passages
README.md - Session API → Query planning: plans and reranks
src/agent/api/main.py - Session API → Search cache: caches searches
src/agent/api/main.py - Session API → Semantic embeddings: requests vectors
src/agent/api/main.py - Semantic embeddings → Embedding endpoint: sends text
README.md - Session API → arXiv search: searches scholarly metadata
README.md - Session API → Answer and claim checks: drafts and verifies
src/agent/api/main.py - Answer and claim checks → Model endpoint: requests model output
src/agent/generation.py - Session API → Evidence catalog: records audit
README.md - Session API → Runtime settings: reads settings
src/agent/api/main.py - Ingestion job → Runtime settings: reads settings
src/agent/ingestion/jobs.py