# inkithai/medimind architecture

> Architecture diagram of the GitHub repository inkithai/medimind, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-08-11). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/inkithai/medimind
- Repository: https://github.com/inkithai/medimind
- Video tour (about a minute): https://gitdiagram.com/inkithai/medimind/video

## Overview

#### 1. Purpose and entry points
MediMind is an anonymous medical-document intelligence workspace. It accepts PDFs/images, extracts structured clinical records, persists a patient snapshot, derives safety/lab insights, and supports cited retrieval Q&A.

Backend entry point: `backend/api.py` FastAPI app under `/api/v1/`. Frontend entry: `frontend/src/main.tsx`, routing/UI rooted in `frontend/src/App.tsx`.

#### 2. Anonymous workspace and access boundary
A browser provisions an anonymous UUID/JWT session, stores credentials locally, and sends both bearer token and user ID on subsequent requests. The API validates identity and scopes every record, object path, and vector collection by user.

Core paths:
- `backend/auth.py`
- `backend/api.py`
- `frontend/src/context/AuthContext.tsx`

Public API surface is health/session creation; document, history, Q&A, conversation, and job endpoints are authenticated.

#### 3. Upload and asynchronous processing orchestration
`POST /documents` accepts multipart files and either processes synchronously or creates an asynchronous parent job. Jobs expose independent per-file states plus batch finalization; a bounded shared worker pool controls provider load and handles partial failures without discarding successful files.

Core paths:
- `backend/api.py`
- `backend/jobs.py`
- `frontend/src/pages/UploadPage.tsx`

The frontend polls jobs and presents extraction progress separately from record finalization (timeline, safety, search indexing).

#### 4. Extraction, validation, and clinical record assembly
The extraction pipeline reads PDF/image content and calls an OpenAI-compatible LLM provider for structured JSON. It supports Groq, Gemini, or generic providers, including multimodal vision models and fallback parsing/retry behavior. Extracted documents are filtered for medical relevance, merged with prior uploads, grouped into a timeline/snapshot, and checked for clinical conflicts.

Core paths:
- `backend/medical_extractor.py`
- `backend/document_filter.py`
- `backend/docs/pipeline.md`

Primary derived outputs are visits, medications, labs, allergies, safety cross-checks, and a structured patient timeline.

#### 5. Persistence and file-storage split
Original files are uploaded to Cloudinary under a user-scoped `mediscan/<user>/` path. Structured documents and patient snapshots are stored in Supabase Postgres. The schema enables RLS and is intended for service-role backend access; raw file bytes and LLM tokens are not stored in the database.

Core paths:
- `backend/storage.py`
- `backend/db.py`
- `backend/supabase_schema.sql`

Material external services: Cloudinary, Supabase, and the selected LLM provider. Runtime secrets and deployment configuration are defined through backend environment variables.

#### 6. Analytics and retrieval boundary
Lab trends are deterministic Python computations over extracted lab values, dates, ranges, and crossings. Retrieval chunks structured timeline content, embeds it using OpenAI embeddings when available or local ONNX fallback, and indexes/searches through a pluggable vector-store layer.

Core paths:
- `backend/lab_trends.py`
- `backend/retrieval.py`
- `backend/vector_store.py`

Vector storage is either local Chroma (requires persistent volume) or Supabase `chunks` (brute-force cosine; deployment-friendly). Q&A self-heals by rebuilding a missing index from persisted documents.

#### 7. Question answering, conversations, and workspace UI
Single-shot Q&A retrieves patient-scoped chunks and returns grounded answers, confidence, source citations, and professional-consultation guidance. In-memory conversations retain turns, rewrite follow-ups into standalone retrieval queries, and summarize old context; they are lost on process restart unless externally persisted.

Core paths:
- `backend/conversation.py`
- `frontend/src/pages/QAPage.tsx`
- `frontend/src/pages/SessionPage.tsx`

The React/Vite frontend renders the patient workspace: dashboard, document viewer, timeline, medicines, lab trends, safety, Q&A, and conversations. It communicates through the centralized client layer at `frontend/src/api/client.ts`.

#### 8. Deployment runtime
Backend runs on Python 3.10+ with Uvicorn/FastAPI and is packaged for Railway via `backend/Procfile` and `backend/Dockerfile`. Frontend is React/TypeScript/Vite, with production hosting configuration in `frontend/vercel.json`. For Railway, Supabase vectors avoid a persistent disk; Chroma requires a mounted volume.

Core paths:
- `backend/Procfile`
- `backend/Dockerfile`
- `frontend/vite.config.ts`

Main entry points: [`backend/api.py`](https://github.com/inkithai/medimind/blob/main/backend/api.py) (REST API), [`backend/docs/pipeline.md`](https://github.com/inkithai/medimind/blob/main/backend/docs/pipeline.md) (Patient record assembly), [`frontend/src/api/client.ts`](https://github.com/inkithai/medimind/blob/main/frontend/src/api/client.ts) (API client), [`backend/jobs.py`](https://github.com/inkithai/medimind/blob/main/backend/jobs.py) (Job orchestration)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_frontend["React workspace"]
  node_app["Workspace UI<br/>React/Vite<br/>[App.tsx]"]
  node_auth_context["Anonymous session<br/>React context<br/>[AuthContext.tsx]"]
  node_api_client["API client<br/>frontend client<br/>[client.ts]"]
  node_upload_ui["Upload &amp; job progress<br/>page<br/>[UploadPage.tsx]"]
  node_qa_ui["Cited Q&amp;A UI<br/>page<br/>[QAPage.tsx]"]
end

subgraph group_backend["FastAPI backend"]
  node_api["REST API<br/>FastAPI<br/>[api.py]"]
  node_auth["Identity scoping<br/>auth middleware<br/>[auth.py]"]
  node_jobs["Job orchestration<br/>worker pool<br/>[jobs.py]"]
  node_extraction["Clinical extraction<br/>LLM pipeline"]
  node_filter["Medical relevance filter<br/>document filter<br/>[document_filter.py]"]
  node_record_assembly["Patient record assembly<br/>clinical pipeline<br/>[pipeline.md]"]
  node_lab_trends["Lab trend analysis<br/>deterministic analytics<br/>[lab_trends.py]"]
  node_qa_conversation["Grounded Q&amp;A<br/>conversation service<br/>[conversation.py]"]
end

subgraph group_data["Storage &amp; retrieval"]
  node_db[("Clinical records &amp; snapshots<br/>Supabase Postgres<br/>[db.py]")]
  node_files[("Original documents<br/>Cloudinary objects<br/>[storage.py]")]
  node_retrieval["Retrieval indexing<br/>embedding pipeline<br/>[retrieval.py]"]
  node_vectors[("Patient vector index<br/>pluggable vector store<br/>[vector_store.py]")]
end

subgraph group_external["External services"]
  node_llm{{"LLM provider<br/>Groq/Gemini/OpenAI-compatible"}}
  node_embedding{{"Embedding provider<br/>OpenAI or local ONNX"}}
end

node_app -->|"uses session"| node_auth_context
node_app -->|"workspace requests"| node_api_client
node_upload_ui -->|"upload and poll jobs"| node_api_client
node_qa_ui -->|"ask questions"| node_api_client
node_api_client -->|"JWT + user ID"| node_api
node_api -->|"authenticate requests"| node_auth
node_api -->|"create/process document jobs"| node_jobs
node_jobs -->|"store originals"| node_files
node_jobs -->|"process each file"| node_extraction
node_extraction -->|"structured extraction"| node_llm
node_extraction -->|"extracted document"| node_filter
node_filter -->|"medical documents"| node_record_assembly
node_record_assembly -->|"persist documents and snapshot"| node_db
node_record_assembly -->|"lab values and dates"| node_lab_trends
node_record_assembly -->|"timeline content"| node_retrieval
node_lab_trends -->|"derived insights"| node_db
node_retrieval -->|"embed chunks"| node_embedding
node_retrieval -->|"index patient chunks"| node_vectors
node_api -->|"Q&amp;A and conversation endpoints"| node_qa_conversation
node_qa_conversation -->|"patient-scoped retrieval"| node_vectors
node_qa_conversation -.->|"rebuild from persisted records"| node_db

click node_app "https://github.com/inkithai/medimind/blob/main/frontend/src/App.tsx"
click node_auth_context "https://github.com/inkithai/medimind/blob/main/frontend/src/context/AuthContext.tsx"
click node_api_client "https://github.com/inkithai/medimind/blob/main/frontend/src/api/client.ts"
click node_upload_ui "https://github.com/inkithai/medimind/blob/main/frontend/src/pages/UploadPage.tsx"
click node_qa_ui "https://github.com/inkithai/medimind/blob/main/frontend/src/pages/QAPage.tsx"
click node_api "https://github.com/inkithai/medimind/blob/main/backend/api.py"
click node_auth "https://github.com/inkithai/medimind/blob/main/backend/auth.py"
click node_jobs "https://github.com/inkithai/medimind/blob/main/backend/jobs.py"
click node_extraction "https://github.com/inkithai/medimind/blob/main/backend/medical_extractor.py"
click node_filter "https://github.com/inkithai/medimind/blob/main/backend/document_filter.py"
click node_record_assembly "https://github.com/inkithai/medimind/blob/main/backend/docs/pipeline.md"
click node_lab_trends "https://github.com/inkithai/medimind/blob/main/backend/lab_trends.py"
click node_qa_conversation "https://github.com/inkithai/medimind/blob/main/backend/conversation.py"
click node_db "https://github.com/inkithai/medimind/blob/main/backend/db.py"
click node_files "https://github.com/inkithai/medimind/blob/main/backend/storage.py"
click node_retrieval "https://github.com/inkithai/medimind/blob/main/backend/retrieval.py"
click node_vectors "https://github.com/inkithai/medimind/blob/main/backend/vector_store.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_app,node_auth_context,node_api_client,node_upload_ui,node_qa_ui toneBlue
class node_api,node_auth,node_jobs,node_extraction,node_filter,node_record_assembly,node_lab_trends,node_qa_conversation toneAmber
class node_db,node_files,node_retrieval,node_vectors toneMint
class node_llm,node_embedding toneRose
```

## Components

### React workspace

- [Workspace UI](https://github.com/inkithai/medimind/blob/main/frontend/src/App.tsx): React/Vite · Dashboard, records, safety, labs, Q&A, and conversations. · `frontend/src/App.tsx`
- [Anonymous session](https://github.com/inkithai/medimind/blob/main/frontend/src/context/AuthContext.tsx): React context · Provisions and locally retains UUID/JWT credentials. · `frontend/src/context/AuthContext.tsx`
- [API client](https://github.com/inkithai/medimind/blob/main/frontend/src/api/client.ts): frontend client · Sends authenticated workspace requests. · `frontend/src/api/client.ts`
- [Upload & job progress](https://github.com/inkithai/medimind/blob/main/frontend/src/pages/UploadPage.tsx): page · Polls per-file extraction and finalization states. · `frontend/src/pages/UploadPage.tsx`
- [Cited Q&A UI](https://github.com/inkithai/medimind/blob/main/frontend/src/pages/QAPage.tsx): page · `frontend/src/pages/QAPage.tsx`

### FastAPI backend

- [REST API](https://github.com/inkithai/medimind/blob/main/backend/api.py): FastAPI · Public health/session endpoints; all workspace data endpoints are authenticated. · `backend/api.py`
- [Identity scoping](https://github.com/inkithai/medimind/blob/main/backend/auth.py): auth middleware · Validates JWT/user identity and scopes records, objects, and vectors. · `backend/auth.py`
- [Job orchestration](https://github.com/inkithai/medimind/blob/main/backend/jobs.py): worker pool · Runs bounded per-file processing and preserves partial successes. · `backend/jobs.py`
- [Clinical extraction](https://github.com/inkithai/medimind/blob/main/backend/medical_extractor.py): LLM pipeline · Parses PDF/images into structured clinical JSON with retries and fallbacks. · `backend/medical_extractor.py`
- [Medical relevance filter](https://github.com/inkithai/medimind/blob/main/backend/document_filter.py): document filter · `backend/document_filter.py`
- [Patient record assembly](https://github.com/inkithai/medimind/blob/main/backend/docs/pipeline.md): clinical pipeline · Merges uploads into timeline, snapshot, safety checks, visits, medications, labs, and allergies. · `backend/docs/pipeline.md`
- [Lab trend analysis](https://github.com/inkithai/medimind/blob/main/backend/lab_trends.py): deterministic analytics · Computes trends, ranges, and threshold crossings. · `backend/lab_trends.py`
- [Grounded Q&A](https://github.com/inkithai/medimind/blob/main/backend/conversation.py): conversation service · Rewrites follow-ups, retrieves citations, and keeps in-memory conversation context. · `backend/conversation.py`

### Storage & retrieval

- [Clinical records & snapshots](https://github.com/inkithai/medimind/blob/main/backend/db.py): Supabase Postgres · RLS-backed structured persistence accessed by the service-role backend. · `backend/db.py`
- [Original documents](https://github.com/inkithai/medimind/blob/main/backend/storage.py): Cloudinary objects · Stored under user-scoped mediscan paths. · `backend/storage.py`
- [Retrieval indexing](https://github.com/inkithai/medimind/blob/main/backend/retrieval.py): embedding pipeline · Chunks timeline content and rebuilds missing indexes from persisted documents. · `backend/retrieval.py`
- [Patient vector index](https://github.com/inkithai/medimind/blob/main/backend/vector_store.py): pluggable vector store · Local Chroma or Supabase chunks, scoped per user. · `backend/vector_store.py`

### External services

- **LLM provider**: Groq/Gemini/OpenAI-compatible · Provides structured extraction and multimodal vision when configured.
- **Embedding provider**: OpenAI or local ONNX

## Connections

- Workspace UI → Anonymous session: uses session
- Workspace UI → API client: workspace requests
- Upload & job progress → API client: upload and poll jobs
- Cited Q&A UI → API client: ask questions
- API client → REST API: JWT + user ID
- REST API → Identity scoping: authenticate requests
- REST API → Job orchestration: create/process document jobs
- Job orchestration → Original documents: store originals
- Job orchestration → Clinical extraction: process each file
- Clinical extraction → LLM provider: structured extraction
- Clinical extraction → Medical relevance filter: extracted document
- Medical relevance filter → Patient record assembly: medical documents
- Patient record assembly → Clinical records & snapshots: persist documents and snapshot
- Patient record assembly → Lab trend analysis: lab values and dates
- Patient record assembly → Retrieval indexing: timeline content
- Lab trend analysis → Clinical records & snapshots: derived insights
- Retrieval indexing → Embedding provider: embed chunks
- Retrieval indexing → Patient vector index: index patient chunks
- REST API → Grounded Q&A: Q&A and conversation endpoints
- Grounded Q&A → Patient vector index: patient-scoped retrieval
- Grounded Q&A → Clinical records & snapshots: rebuild from persisted records