How rishabh-108272/multi-agentic-rag-for-biomarker-validation works
1. Top-level shape
- The repo is split into two primary runtime boundaries:
- backend/: a Django-based API and analysis engine.
- project/: a React + TypeScript client built with Vite.
- The frontend is mostly a presentation and orchestration layer; the backend owns the biomedical pipeline, persistence, and async work.
2. User-facing flow
- The main user journey starts on project/src/pages/HomePage.tsx, where a CSV patient file is uploaded and analysis is initiated.
- project/src/pages/ResultsPage.tsx then polls for progress and renders the outputs once backend processing completes.
- The UI is decomposed into result-specific components such as subtype cards, gene/drug tables, agent reports, summary reports, and a progress view.
3. Frontend integration boundary
- project/src/lib/api.ts is the client’s main HTTP bridge to the backend REST endpoints.
- project/src/lib/supabase.ts plus the project/supabase/migrations/ folder indicate a secondary persistence/integration layer on the frontend side, likely for analysis history or dashboard data.
- The frontend is therefore not just static display; it coordinates upload, status polling, and result retrieval from backend services.
4. Backend API layer
- The backend Django project lives under backend/lung_cancer_api/, with routing in urls.py and HTTP entry points in views.py.
- The backend/pipeline/views.py and backend/pipeline/urls.py modules expose the analysis API surface: upload, status, results, summary retrieval, and cleanup.
- This API layer is the external contract the frontend depends on.
5. Async orchestration
- The heavy analysis is not handled inline with the request; it is orchestrated through Celery in backend/lung_cancer_api/celery.py, backend/pipeline/tasks.py, and backend/pipeline/queue_tasks.py.
- This is the main scalability boundary: request/response traffic stays in Django, while long-running work is queued and processed separately.
- The presence of celerybeat-schedule* files suggests scheduled/background execution is part of the runtime model as well.
6. Core analysis pipeline
- The pipeline is a staged workflow that transforms raw CSV input into layered biomedical interpretation.
- The stages visible in the repo are:
1. input normalization,
2. subtype classification,
3. explainability and feature ranking,
4. gene mapping,
5. drug candidate discovery,
6. multi-agent evidence validation,
7. final summary synthesis.
- backend/pipeline/tasks.py is the orchestration center for these steps.
7. Subtype classification subsystem
- backend/pipeline/ml_classifier.py handles subtype prediction and confidence scoring.
- The README indicates the implementation can call remote Hugging Face Spaces endpoints, so classification is effectively an external model service dependency rather than only local inference.
- The current system supports at least lung and colorectal analysis flows.
8. Explainability and gene interpretation
- backend/pipeline/xai_engine.py generates feature importance and explanation artifacts, likely using SHAP/LIME-style logic with fallback strategies for high-dimensional data.
- backend/pipeline/gene_mapper.py and backend/pipeline/gene_index_map.json convert opaque model features into biologically interpretable gene-level entities.
- This is an important semantic boundary: model outputs become domain-readable markers before downstream drug and evidence analysis.
9. Drug candidate subsystem
- backend/pipeline/drug_engine.py links genes to possible therapeutic options, using interaction data such as DGIdb-derived associations.
- The output of this stage is not a treatment recommendation engine by itself; it is a ranked candidate list that feeds later validation and reporting.
- In architecture terms, this is the bridge from biomarker interpretation to therapeutic exploration.
10. Multi-agent RAG validation layer
- backend/pipeline/langgraph_pipeline.py is the reasoning layer that validates and enriches intermediate results using a multi-agent RAG approach.
- The README describes this stage as assembling evidence from biomedical literature and domain knowledge sources, then consolidating it into interpretable reports.
- This is the system’s main “validation boundary”: predictions and drug suggestions are cross-checked against external evidence rather than presented alone.
11. Persistence model
- backend/pipeline/models.py defines persisted analysis entities such as PatientAnalysis, XAIResult, DrugCandidate, AgentReport, and PatientSummaryReport.
- These models capture the full lifecycle of a run: metadata, intermediate artifacts, agent outputs, and final summaries.
- The backend uses Django migrations and, by default, SQLite, which makes persistence local and simple unless overridden in deployment.
12. Result composition
- The final response to the client is assembled from multiple backend outputs rather than a single monolithic score.
- The frontend renders this as separate sections: subtype prediction, driver genes, drug candidates, agent reports, and a summary narrative.
- That separation mirrors the backend data model and makes the system well-suited for diagramming as a pipeline with fan-out/fan-in stages.
13. Deployment/runtime boundaries
- The backend includes a Dockerfile, requirements.txt, and runtime.txt, indicating it is designed for containerized or platform-managed Python deployment.
- The frontend includes vercel.json, showing it is intended for Vercel-style static hosting.
- The overall architecture is therefore a decoupled web client plus a compute-heavy Python service with async workers and external biomedical APIs.
14. What to emphasize in an architecture diagram
- Show the frontend as the entry point for upload and visualization.
- Show the Django API as the control plane, Celery as the async execution plane, and the pipeline modules as sequential processing blocks.
- Show external dependencies on Hugging Face model services, DGIdb, literature/evidence sources, and the LLM-backed agent layer.
- Show persistence as a shared backend datastore for analyses, explanation artifacts, drug candidates, and generated reports.
Main entry points: tasks.py Pipeline tasks · ResultsPage.tsx Results page · lung_cancer_api Django API · langgraph_pipeline.py RAG agents
Components 23
Frontend
React + TypeScript client in project/
- Home page · Upload and analysis start screen
project/src/pages/HomePage.tsx - Results page · Polls status and renders completed analysis
project/src/pages/ResultsPage.tsx - API client · Frontend HTTP layer for backend calls
project/src/lib/api.ts - Supabase client · Secondary persistence and dashboard integration
project/src/lib/supabase.ts - Upload box · Patient CSV upload control
project/src/components/UploadBox.tsx - Progress view · Shows analysis progress and polling state
project/src/components/PipelineProgress.tsx - Result views · Subtype, genes, drugs, reports, and summary panels
project/src/components/SubtypeCard.tsx
Backend
Django API, async workers, and analysis pipeline in backend/
- Django API · Main backend project and HTTP control plane
backend/lung_cancer_api - API routes · Backend URL routing and entrypoints
backend/lung_cancer_api/urls.py - API views · Backend request handlers and response assembly
backend/lung_cancer_api/views.py - Pipeline API · Upload, status, results, summary, and cleanup endpoints
backend/pipeline/views.py - Celery · Background job coordination and scheduling
backend/lung_cancer_api/celery.py - Pipeline tasks · Coordinates the staged biomarker analysis workflow
backend/pipeline/tasks.py - Queue tasks · Queued task implementations and dispatch helpers
backend/pipeline/queue_tasks.py - Classifier · Subtype prediction and confidence scoring
backend/pipeline/ml_classifier.py - XAI engine · Feature importance and explanation generation
backend/pipeline/xai_engine.py - Gene mapper · Maps model features to gene-level entities
backend/pipeline/gene_mapper.py - Drug engine · Links genes to therapeutic candidates
backend/pipeline/drug_engine.py - RAG agents · Multi-agent literature-backed validation and synthesis
backend/pipeline/langgraph_pipeline.py - Data models · Persisted analyses, artifacts, candidates, and reports
backend/pipeline/models.py
External services
Model, evidence, and persistence integrations
- HF models · Remote subtype inference service dependency
- Evidence sources · Biomedical literature and knowledge sources used by agents
- Supabase · Frontend-side persistence/integration backend
Connections 23
- Home page → Upload box: uploads CSV · User starts analysis by uploading patient data
- Home page → API client: start analysis · Frontend initiates backend analysis request
- Results page → Progress view: polls status · Results view tracks backend progress
- Results page → Result views: renders outputs · Displays subtype, genes, drugs, reports, and summary
- API client → Django API: REST calls · Frontend communicates with backend API
- Supabase client → Supabase: syncs data · Client persistence and dashboard integration
- Django API → API routes: routes · Django request routing
- API routes → API views: dispatches · Routes requests to HTTP handlers
- API views → Pipeline API: exposes · Pipeline-specific analysis endpoints
- Pipeline API → Celery: queues work · Long-running analysis is handed off asynchronously
- Celery → Pipeline tasks: runs workflow · Celery executes staged pipeline orchestration
- Pipeline tasks → Queue tasks: dispatches jobs · Pipeline splits work into queued stages
- Pipeline tasks → Classifier: classifies subtype · Predicts cancer subtype from input data
- Pipeline tasks → XAI engine: explains features · Generates explainability artifacts
- XAI engine → Gene mapper: maps genes · Converts model features to biological entities
- Gene mapper → Drug engine: feeds markers · Gene-level interpretation drives therapeutic lookup
- Drug engine → RAG agents: validates candidates · Candidate drugs are checked and enriched by agents
- RAG agents → Evidence sources: consults · Agents retrieve biomedical evidence and context
- Classifier → HF models: remote inference · Subtype model can call external Hugging Face services
- RAG agents → Data models: persists reports · Agent outputs and synthesized summaries are stored
- Pipeline tasks → Data models: persists run data · Analysis metadata and intermediate artifacts are saved
- Django API → Data models: reads/writes · API layer accesses persisted analysis state
- Results page → Supabase client: dashboard sync · Frontend may read/write analysis history