# hoomesh0407/oralsenseai architecture

> Architecture diagram of the GitHub repository hoomesh0407/oralsenseai, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-10-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hoomesh0407/oralsenseai
- Repository: https://github.com/hoomesh0407/oralsenseai

## Overview

OralSense AI is a research prototype for classifying oral squamous cell carcinoma histopathology into three grades. Its main user workflow is image upload through the web interface, API validation and preprocessing, inference by one or all four registered models, and a response containing the predicted grade, probabilities, and latency. The README also highlights Grad-CAM explanations and model evaluation. The graph uses the sampled `backend/app` implementation; the tree contains a parallel `app/backend/app` copy. Research and explanation capabilities are included, but their runtime wiring was not shown.

Main entry points: [`backend/app/model_factory.py`](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/model_factory.py) (Model factory), [`backend/app/main.py`](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/main.py) (FastAPI application), [`backend/app/inference.py`](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/inference.py) (Inference orchestration), [`backend/app/model_registry.py`](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/model_registry.py) (Model registry)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_interface["Web interface and API"]
  node_frontend["Interactive dashboard<br/>[app.js]"]
  node_api["FastAPI application<br/>[main.py]"]
  node_prediction_routes["Prediction endpoints<br/>[predict.py]"]
  node_health_routes["Health endpoint<br/>[health.py]"]
  node_model_routes["Model listing<br/>[models.py]"]
  node_schemas["API response schemas<br/>[schemas.py]"]
end

subgraph group_inference["Inference and model selection"]
  node_inference["Inference orchestration<br/>[inference.py]"]
  node_image_utils["Image loading and transforms<br/>[image_utils.py]"]
  node_device["Device selection<br/>[device_utils.py]"]
  node_registry["Model registry<br/>[model_registry.py]"]
  node_factory["Model factory<br/>[model_factory.py]"]
  node_config["Model configuration<br/>[config.py]"]
  node_checkpoint["Checkpoint loading"]
end

subgraph group_architectures["Classification models"]
  node_densenet["DenseNet121<br/>[densenet121.py]"]
  node_medsiglip["MedSigLIP<br/>[medsiglip.py]"]
  node_lora["MedSigLIP LoRA<br/>[medsiglip_lora.py]"]
  node_fusion["ViT hybrid fusion<br/>[vit_fusion.py]"]
end

subgraph group_research["Research and explainability"]
  node_training["Model training<br/>[train.py]"]
  node_evaluation["Model evaluation<br/>[evaluate.py]"]
  node_gradcam["Grad-CAM explainability<br/>[gradcam_utils.py]"]
end

node_user(("Image uploader"))
node_research_user(("Researcher"))

node_user -.->|"uploads image"| node_frontend
node_frontend -.->|"requests predictions"| node_api
node_api -->|"registers route"| node_prediction_routes
node_api -->|"registers route"| node_health_routes
node_api -->|"registers route"| node_model_routes
node_api -->|"loads models at startup"| node_registry
node_prediction_routes -->|"runs inference"| node_inference
node_prediction_routes -->|"loads uploaded image"| node_image_utils
node_prediction_routes -->|"validates response"| node_schemas
node_model_routes -->|"lists model metadata"| node_registry
node_health_routes -->|"checks registry"| node_registry
node_health_routes -->|"reports device"| node_device
node_inference -->|"gets model"| node_registry
node_inference -->|"preprocesses image"| node_image_utils
node_inference -->|"selects device"| node_device
node_inference -->|"uses class names"| node_config
node_registry -->|"creates models"| node_factory
node_factory -->|"reads model settings"| node_config
node_factory -->|"constructs model"| node_densenet
node_factory -->|"constructs model"| node_medsiglip
node_factory -->|"constructs model"| node_lora
node_factory -->|"constructs model"| node_fusion
node_medsiglip -->|"loads checkpoint"| node_checkpoint
node_medsiglip -->|"selects device"| node_device
node_lora -->|"loads checkpoint"| node_checkpoint
node_lora -->|"selects device"| node_device
node_research_user -.->|"trains models"| node_training
node_research_user -.->|"evaluates models"| node_evaluation
node_research_user -.->|"generates explanations"| node_gradcam
node_frontend -.->|"displays overlays"| node_gradcam

click node_frontend "https://github.com/hoomesh0407/oralsenseai/blob/main/frontend/app.js"
click node_api "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/main.py"
click node_prediction_routes "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/routes/predict.py"
click node_health_routes "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/routes/health.py"
click node_model_routes "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/routes/models.py"
click node_schemas "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/schemas.py"
click node_inference "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/inference.py"
click node_image_utils "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/utils/image_utils.py"
click node_device "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/utils/device_utils.py"
click node_registry "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/model_registry.py"
click node_factory "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/model_factory.py"
click node_config "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/config.py"
click node_densenet "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/models/densenet121.py"
click node_medsiglip "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/models/medsiglip.py"
click node_lora "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/models/medsiglip_lora.py"
click node_fusion "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/models/vit_fusion.py"
click node_checkpoint "https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/utils/checkpoint_utils.py"
click node_training "https://github.com/hoomesh0407/oralsenseai/blob/main/train.py"
click node_evaluation "https://github.com/hoomesh0407/oralsenseai/blob/main/evaluate.py"
click node_gradcam "https://github.com/hoomesh0407/oralsenseai/blob/main/src/utils/gradcam_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_frontend,node_api,node_prediction_routes,node_health_routes,node_model_routes,node_schemas toneBlue
class node_inference,node_image_utils,node_device,node_registry,node_factory,node_config,node_checkpoint toneAmber
class node_densenet,node_medsiglip,node_lora,node_fusion toneMint
class node_training,node_evaluation,node_gradcam toneRose
class node_user,node_research_user toneIndigo
```

## Components

### Web interface and API

- [Interactive dashboard](https://github.com/hoomesh0407/oralsenseai/blob/main/frontend/app.js): `frontend/app.js`
- [FastAPI application](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/main.py): `backend/app/main.py`
- [Prediction endpoints](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/routes/predict.py): `backend/app/routes/predict.py`
- [Health endpoint](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/routes/health.py): `backend/app/routes/health.py`
- [Model listing](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/routes/models.py): `backend/app/routes/models.py`
- [API response schemas](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/schemas.py): `backend/app/schemas.py`

### Inference and model selection

- [Inference orchestration](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/inference.py): `backend/app/inference.py`
- [Image loading and transforms](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/utils/image_utils.py): `backend/app/utils/image_utils.py`
- [Device selection](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/utils/device_utils.py): `backend/app/utils/device_utils.py`
- [Model registry](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/model_registry.py): `backend/app/model_registry.py`
- [Model factory](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/model_factory.py): `backend/app/model_factory.py`
- [Model configuration](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/config.py): `backend/app/config.py`
- [Checkpoint loading](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/utils/checkpoint_utils.py): `backend/app/utils/checkpoint_utils.py`

### Classification models

- [DenseNet121](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/models/densenet121.py): `backend/app/models/densenet121.py`
- [MedSigLIP](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/models/medsiglip.py): `backend/app/models/medsiglip.py`
- [MedSigLIP LoRA](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/models/medsiglip_lora.py): `backend/app/models/medsiglip_lora.py`
- [ViT hybrid fusion](https://github.com/hoomesh0407/oralsenseai/blob/main/backend/app/models/vit_fusion.py): `backend/app/models/vit_fusion.py`

### Research and explainability

- [Model training](https://github.com/hoomesh0407/oralsenseai/blob/main/train.py): `train.py`
- [Model evaluation](https://github.com/hoomesh0407/oralsenseai/blob/main/evaluate.py): `evaluate.py`
- [Grad-CAM explainability](https://github.com/hoomesh0407/oralsenseai/blob/main/src/utils/gradcam_utils.py): `src/utils/gradcam_utils.py`

### Other components

- **Image uploader**
- **Researcher**

## Connections

- Image uploader → Interactive dashboard: uploads image (evidence: `README.md`)
- Interactive dashboard → FastAPI application: requests predictions (evidence: `README.md`)
- FastAPI application → Prediction endpoints: registers route (evidence: `backend/app/main.py`)
- FastAPI application → Health endpoint: registers route (evidence: `backend/app/main.py`)
- FastAPI application → Model listing: registers route (evidence: `backend/app/main.py`)
- FastAPI application → Model registry: loads models at startup (evidence: `backend/app/main.py`)
- Prediction endpoints → Inference orchestration: runs inference (evidence: `backend/app/routes/predict.py`)
- Prediction endpoints → Image loading and transforms: loads uploaded image (evidence: `backend/app/routes/predict.py`)
- Prediction endpoints → API response schemas: validates response (evidence: `backend/app/routes/predict.py`)
- Model listing → Model registry: lists model metadata (evidence: `backend/app/routes/models.py`)
- Health endpoint → Model registry: checks registry (evidence: `backend/app/routes/health.py`)
- Health endpoint → Device selection: reports device (evidence: `backend/app/routes/health.py`)
- Inference orchestration → Model registry: gets model (evidence: `backend/app/inference.py`)
- Inference orchestration → Image loading and transforms: preprocesses image (evidence: `backend/app/inference.py`)
- Inference orchestration → Device selection: selects device (evidence: `backend/app/inference.py`)
- Inference orchestration → Model configuration: uses class names (evidence: `backend/app/inference.py`)
- Model registry → Model factory: creates models (evidence: `backend/app/model_registry.py`)
- Model factory → Model configuration: reads model settings (evidence: `backend/app/model_factory.py`)
- Model factory → DenseNet121: constructs model (evidence: `backend/app/model_factory.py`)
- Model factory → MedSigLIP: constructs model (evidence: `backend/app/model_factory.py`)
- Model factory → MedSigLIP LoRA: constructs model (evidence: `backend/app/model_factory.py`)
- Model factory → ViT hybrid fusion: constructs model (evidence: `backend/app/model_factory.py`)
- MedSigLIP → Checkpoint loading: loads checkpoint (evidence: `backend/app/models/medsiglip.py`)
- MedSigLIP → Device selection: selects device (evidence: `backend/app/models/medsiglip.py`)
- MedSigLIP LoRA → Checkpoint loading: loads checkpoint (evidence: `backend/app/models/medsiglip_lora.py`)
- MedSigLIP LoRA → Device selection: selects device (evidence: `backend/app/models/medsiglip_lora.py`)
- Researcher → Model training: trains models
- Researcher → Model evaluation: evaluates models
- Researcher → Grad-CAM explainability: generates explanations (evidence: `README.md`)
- Interactive dashboard → Grad-CAM explainability: displays overlays (evidence: `README.md`)
