# ifsaurabh/crossscan_multimodal_agentic_rag architecture

> Architecture diagram of the GitHub repository ifsaurabh/crossscan_multimodal_agentic_rag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-06). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/ifsaurabh/crossscan_multimodal_agentic_rag
- Repository: https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag

## Overview

CrossScan is a multimodal RAG application: uploaded research papers are validated, transformed into searchable text and image data, then retrieved to produce cited answers. The README documents the ingestion stages and user-facing capabilities; the SOURCE INDEX and excerpts support key implementation links. Several retrieval branches and UI integrations are unsampled, so their connections are omitted or marked unverified.

Main entry points: [`src/retrieval/retrieval_graph.py`](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/retrieval_graph.py) (Retrieval workflow), [`src/ingestion/pipeline.py`](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/pipeline.py) (Ingestion pipeline), [`src/ingestion/ingest_worker.py`](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/ingest_worker.py) (Ingestion worker), [`src/retrieval/chat_app.py`](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/chat_app.py) (Chat interface)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_ingestion["Document ingestion"]
  node_ingest_trigger["Ingest trigger<br/>[main.py]"]
  node_worker["Ingestion worker<br/>[ingest_worker.py]"]
  node_intake["Intake checks<br/>[intake_check.py]"]
  node_extract["Document extraction<br/>[extraction.py]"]
  node_prepare["Section preparation"]
  node_guardrails["Ingestion guardrails"]
  node_chunk["Parent-child chunking<br/>[chunk_documents.py]"]
  node_embeddings["Text and image embeddings"]
  node_pipeline["Ingestion pipeline<br/>[pipeline.py]"]
end

subgraph group_retrieval["Question answering"]
  node_query_entry["Query entry<br/>[run_query.py]"]
  node_graph["Retrieval workflow<br/>[retrieval_graph.py]"]
  node_planner["Query planning"]
  node_executor["Search execution"]
  node_cache[("Answer cache<br/>[query_cache.py]")]
  node_quality["Answer generation"]
  node_output_checks["Citation and safety checks"]
  node_media["Table and image context<br/>[table_context.py]"]
end

subgraph group_product["User experience"]
  node_chat_app["Chat interface<br/>[chat_app.py]"]
  node_auth["Authentication and users<br/>[auth.py]"]
  node_evaluation["Offline evaluation<br/>[run_evaluation.py]"]
end

subgraph group_platform["Shared services"]
  node_query_safety["Query safety<br/>[query_guardrail.py]"]
  node_llm["Language model routing<br/>[llm_connection.py]"]
  node_observability["Tracing and usage<br/>[tracing.py]"]
end

node_uploader(("Paper uploader"))
node_cloud[("Cloud Storage")]
node_queue["Pub/Sub queue"]
node_postgres[("Postgres / pgvector")]
node_user(("Chat user"))
node_external_models{{"Embedding models"}}
node_langfuse{{"Langfuse"}}

node_uploader -->|"uploads papers"| node_cloud
node_cloud -->|"notifies upload"| node_queue
node_queue -->|"delivers upload"| node_worker
node_worker -->|"checks document"| node_intake
node_worker -->|"processes accepted file"| node_pipeline
node_worker -->|"moves file"| node_cloud
node_pipeline -->|"extracts content"| node_extract
node_pipeline -->|"prepares sections"| node_prepare
node_pipeline -->|"redacts and flags"| node_guardrails
node_pipeline -->|"creates chunks"| node_chunk
node_pipeline -->|"embeds content"| node_embeddings
node_pipeline -->|"replaces document transactionally"| node_postgres
node_embeddings -->|"uses text and image models"| node_external_models
node_pipeline -->|"stores extracted images"| node_cloud
node_user -->|"asks questions"| node_chat_app
node_chat_app -.->|"supports sign-in"| node_auth
node_chat_app -.->|"submits query"| node_query_entry
node_query_entry -->|"invokes workflow"| node_graph
node_graph -->|"checks query"| node_query_safety
node_graph -->|"checks cached answer"| node_cache
node_graph -->|"routes query"| node_planner
node_graph -->|"runs retrieval"| node_executor
node_graph -->|"generates response"| node_quality
node_graph -->|"checks answer"| node_output_checks
node_graph -->|"attaches context"| node_media
node_executor -.->|"searches vectors"| node_postgres
node_quality -.->|"requests generation"| node_llm
node_graph -.->|"reads and writes retrieval data"| node_postgres
node_chat_app -.->|"uses user and quota data"| node_postgres
node_query_entry -->|"records timings"| node_observability
node_observability -.->|"sends traces"| node_langfuse
node_evaluation -->|"evaluates query results"| node_query_entry
node_evaluation -->|"measures citations"| node_output_checks

click node_ingest_trigger "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/functions/ingest_trigger/main.py"
click node_worker "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/ingest_worker.py"
click node_intake "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/intake_check.py"
click node_extract "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/extraction.py"
click node_prepare "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/prepare_documents.py"
click node_guardrails "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/ingestion_guardrails.py"
click node_chunk "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/chunk_documents.py"
click node_embeddings "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/tree/main/src/ingestion"
click node_pipeline "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/pipeline.py"
click node_chat_app "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/chat_app.py"
click node_auth "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/auth.py"
click node_query_entry "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/run_query.py"
click node_graph "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/retrieval_graph.py"
click node_planner "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/agent_transform_route.py"
click node_executor "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/retrieval_executor.py"
click node_cache "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/query_cache.py"
click node_quality "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/agent_quality_generate.py"
click node_output_checks "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/output_guardrail.py"
click node_media "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/table_context.py"
click node_query_safety "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/shared/query_guardrail.py"
click node_llm "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/shared/llm_connection.py"
click node_observability "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/tracing.py"
click node_evaluation "https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/evaluation/run_evaluation.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_ingest_trigger,node_worker,node_intake,node_extract,node_prepare,node_guardrails,node_chunk,node_embeddings,node_pipeline,node_user toneBlue
class node_query_entry,node_graph,node_planner,node_executor,node_cache,node_quality,node_output_checks,node_media,node_cloud,node_postgres toneAmber
class node_chat_app,node_auth,node_evaluation toneMint
class node_query_safety,node_llm,node_observability,node_queue toneRose
class node_uploader,node_external_models,node_langfuse toneIndigo
```

## Components

### Document ingestion

- [Ingest trigger](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/functions/ingest_trigger/main.py): `functions/ingest_trigger/main.py`
- [Ingestion worker](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/ingest_worker.py): `src/ingestion/ingest_worker.py`
- [Intake checks](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/intake_check.py): `src/ingestion/intake_check.py`
- [Document extraction](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/extraction.py): `src/ingestion/extraction.py`
- [Section preparation](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/prepare_documents.py): `src/ingestion/prepare_documents.py`
- [Ingestion guardrails](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/ingestion_guardrails.py): `src/ingestion/ingestion_guardrails.py`
- [Parent-child chunking](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/chunk_documents.py): `src/ingestion/chunk_documents.py`
- [Text and image embeddings](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/tree/main/src/ingestion): `src/ingestion`
- [Ingestion pipeline](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/ingestion/pipeline.py): `src/ingestion/pipeline.py`

### Question answering

- [Query entry](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/run_query.py): `src/retrieval/run_query.py`
- [Retrieval workflow](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/retrieval_graph.py): `src/retrieval/retrieval_graph.py`
- [Query planning](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/agent_transform_route.py): `src/retrieval/agent_transform_route.py`
- [Search execution](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/retrieval_executor.py): `src/retrieval/retrieval_executor.py`
- [Answer cache](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/query_cache.py): `src/retrieval/query_cache.py`
- [Answer generation](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/agent_quality_generate.py): `src/retrieval/agent_quality_generate.py`
- [Citation and safety checks](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/output_guardrail.py): `src/retrieval/output_guardrail.py`
- [Table and image context](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/table_context.py): `src/retrieval/table_context.py`

### User experience

- [Chat interface](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/chat_app.py): `src/retrieval/chat_app.py`
- [Authentication and users](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/auth.py): `src/retrieval/auth.py`
- [Offline evaluation](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/evaluation/run_evaluation.py): `src/evaluation/run_evaluation.py`

### Shared services

- [Query safety](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/shared/query_guardrail.py): `src/shared/query_guardrail.py`
- [Language model routing](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/shared/llm_connection.py): `src/shared/llm_connection.py`
- [Tracing and usage](https://github.com/ifsaurabh/crossscan_multimodal_agentic_rag/blob/main/src/retrieval/tracing.py): `src/retrieval/tracing.py`

### Other components

- **Paper uploader**
- **Cloud Storage**
- **Pub/Sub queue**
- **Postgres / pgvector**
- **Chat user**
- **Embedding models**
- **Langfuse**

## Connections

- Paper uploader → Cloud Storage: uploads papers
- Cloud Storage → Pub/Sub queue: notifies upload
- Pub/Sub queue → Ingestion worker: delivers upload (evidence: `src/ingestion/ingest_worker.py`)
- Ingestion worker → Intake checks: checks document (evidence: `src/ingestion/ingest_worker.py`)
- Ingestion worker → Ingestion pipeline: processes accepted file (evidence: `src/ingestion/ingest_worker.py`)
- Ingestion worker → Cloud Storage: moves file
- Ingestion pipeline → Document extraction: extracts content (evidence: `src/ingestion/pipeline.py`)
- Ingestion pipeline → Section preparation: prepares sections (evidence: `src/ingestion/pipeline.py`)
- Ingestion pipeline → Ingestion guardrails: redacts and flags (evidence: `src/ingestion/pipeline.py`)
- Ingestion pipeline → Parent-child chunking: creates chunks (evidence: `src/ingestion/pipeline.py`)
- Ingestion pipeline → Text and image embeddings: embeds content (evidence: `src/ingestion/pipeline.py`)
- Ingestion pipeline → Postgres / pgvector: replaces document transactionally
- Text and image embeddings → Embedding models: uses text and image models
- Ingestion pipeline → Cloud Storage: stores extracted images
- Chat user → Chat interface: asks questions
- Chat interface → Authentication and users: supports sign-in
- Chat interface → Query entry: submits query
- Query entry → Retrieval workflow: invokes workflow (evidence: `src/retrieval/run_query.py`)
- Retrieval workflow → Query safety: checks query (evidence: `src/retrieval/retrieval_graph.py`)
- Retrieval workflow → Answer cache: checks cached answer (evidence: `src/retrieval/retrieval_graph.py`)
- Retrieval workflow → Query planning: routes query (evidence: `src/retrieval/retrieval_graph.py`)
- Retrieval workflow → Search execution: runs retrieval (evidence: `src/retrieval/retrieval_graph.py`)
- Retrieval workflow → Answer generation: generates response (evidence: `src/retrieval/retrieval_graph.py`)
- Retrieval workflow → Citation and safety checks: checks answer (evidence: `src/retrieval/retrieval_graph.py`)
- Retrieval workflow → Table and image context: attaches context (evidence: `src/retrieval/retrieval_graph.py`)
- Search execution → Postgres / pgvector: searches vectors
- Answer generation → Language model routing: requests generation
- Retrieval workflow → Postgres / pgvector: reads and writes retrieval data
- Chat interface → Postgres / pgvector: uses user and quota data
- Query entry → Tracing and usage: records timings (evidence: `src/retrieval/run_query.py`)
- Tracing and usage → Langfuse: sends traces
- Offline evaluation → Query entry: evaluates query results (evidence: `src/evaluation/run_evaluation.py`)
- Offline evaluation → Citation and safety checks: measures citations (evidence: `src/evaluation/run_evaluation.py`)