How ifsaurabh/crossscan_multimodal_agentic_rag works
CrossScan is a multimodal RAG application: uploaded research papers are validated, transformed into searchable text and image data, then retrieved to produce cited answers. The README documents the ingestion stages and user-facing capabilities; the SOURCE INDEX and excerpts support key implementation links. Several retrieval branches and UI integrations are unsampled, so their connections are omitted or marked unverified.
Main entry points: retrieval_graph.py Retrieval workflow · pipeline.py Ingestion pipeline · ingest_worker.py Ingestion worker · chat_app.py Chat interface
Components 30
Document ingestion
- Ingest trigger
functions/ingest_trigger/main.py - Ingestion worker
src/ingestion/ingest_worker.py - Intake checks
src/ingestion/intake_check.py - Document extraction
src/ingestion/extraction.py - Section preparation
src/ingestion/prepare_documents.py - Ingestion guardrails
src/ingestion/ingestion_guardrails.py - Parent-child chunking
src/ingestion/chunk_documents.py - Text and image embeddings
src/ingestion - Ingestion pipeline
src/ingestion/pipeline.py
Question answering
- Query entry
src/retrieval/run_query.py - Retrieval workflow
src/retrieval/retrieval_graph.py - Query planning
src/retrieval/agent_transform_route.py - Search execution
src/retrieval/retrieval_executor.py - Answer cache
src/retrieval/query_cache.py - Answer generation
src/retrieval/agent_quality_generate.py - Citation and safety checks
src/retrieval/output_guardrail.py - Table and image context
src/retrieval/table_context.py
User experience
- Chat interface
src/retrieval/chat_app.py - Authentication and users
src/retrieval/auth.py - Offline evaluation
src/evaluation/run_evaluation.py
Shared services
- Query safety
src/shared/query_guardrail.py - Language model routing
src/shared/llm_connection.py - Tracing and usage
src/retrieval/tracing.py
Other components
- Paper uploader
- Cloud Storage
- Pub/Sub queue
- Postgres / pgvector
- Chat user
- Embedding models
- Langfuse
Connections 33
- Paper uploader → Cloud Storage: uploads papers
- Cloud Storage → Pub/Sub queue: notifies upload
- Pub/Sub queue → Ingestion worker: delivers upload
src/ingestion/ingest_worker.py - Ingestion worker → Intake checks: checks document
src/ingestion/ingest_worker.py - Ingestion worker → Ingestion pipeline: processes accepted file
src/ingestion/ingest_worker.py - Ingestion worker → Cloud Storage: moves file
- Ingestion pipeline → Document extraction: extracts content
src/ingestion/pipeline.py - Ingestion pipeline → Section preparation: prepares sections
src/ingestion/pipeline.py - Ingestion pipeline → Ingestion guardrails: redacts and flags
src/ingestion/pipeline.py - Ingestion pipeline → Parent-child chunking: creates chunks
src/ingestion/pipeline.py - Ingestion pipeline → Text and image embeddings: embeds content
src/ingestion/pipeline.py - Ingestion pipeline → Postgres / pgvector: replaces document transactionally
- Text and image embeddings → Embedding models: uses text and image models
- Ingestion pipeline → Cloud Storage: stores extracted images
- Chat user → Chat interface: asks questions
- Chat interface → Authentication and users: supports sign-in
- Chat interface → Query entry: submits query
- Query entry → Retrieval workflow: invokes workflow
src/retrieval/run_query.py - Retrieval workflow → Query safety: checks query
src/retrieval/retrieval_graph.py - Retrieval workflow → Answer cache: checks cached answer
src/retrieval/retrieval_graph.py - Retrieval workflow → Query planning: routes query
src/retrieval/retrieval_graph.py - Retrieval workflow → Search execution: runs retrieval
src/retrieval/retrieval_graph.py - Retrieval workflow → Answer generation: generates response
src/retrieval/retrieval_graph.py - Retrieval workflow → Citation and safety checks: checks answer
src/retrieval/retrieval_graph.py - Retrieval workflow → Table and image context: attaches context
src/retrieval/retrieval_graph.py - Search execution → Postgres / pgvector: searches vectors
- Answer generation → Language model routing: requests generation
- Retrieval workflow → Postgres / pgvector: reads and writes retrieval data
- Chat interface → Postgres / pgvector: uses user and quota data
- Query entry → Tracing and usage: records timings
src/retrieval/run_query.py - Tracing and usage → Langfuse: sends traces
- Offline evaluation → Query entry: evaluates query results
src/evaluation/run_evaluation.py - Offline evaluation → Citation and safety checks: measures citations
src/evaluation/run_evaluation.py