How hoomesh0407/oralsenseai works
OralSense AI is a research prototype for classifying oral squamous cell carcinoma histopathology into three grades. Its main user workflow is image upload through the web interface, API validation and preprocessing, inference by one or all four registered models, and a response containing the predicted grade, probabilities, and latency. The README also highlights Grad-CAM explanations and model evaluation. The graph uses the sampled backend/app implementation; the tree contains a parallel app/backend/app copy. Research and explanation capabilities are included, but their runtime wiring was not shown.
Main entry points: model_factory.py Model factory · main.py FastAPI application · inference.py Inference orchestration · model_registry.py Model registry
Components 22
Web interface and API
- Interactive dashboard
frontend/app.js - FastAPI application
backend/app/main.py - Prediction endpoints
backend/app/routes/predict.py - Health endpoint
backend/app/routes/health.py - Model listing
backend/app/routes/models.py - API response schemas
backend/app/schemas.py
Inference and model selection
- Inference orchestration
backend/app/inference.py - Image loading and transforms
backend/app/utils/image_utils.py - Device selection
backend/app/utils/device_utils.py - Model registry
backend/app/model_registry.py - Model factory
backend/app/model_factory.py - Model configuration
backend/app/config.py - Checkpoint loading
backend/app/utils/checkpoint_utils.py
Classification models
- DenseNet121
backend/app/models/densenet121.py - MedSigLIP
backend/app/models/medsiglip.py - MedSigLIP LoRA
backend/app/models/medsiglip_lora.py - ViT hybrid fusion
backend/app/models/vit_fusion.py
Research and explainability
- Model training
train.py - Model evaluation
evaluate.py - Grad-CAM explainability
src/utils/gradcam_utils.py
Other components
- Image uploader
- Researcher
Connections 30
- Image uploader → Interactive dashboard: uploads image
README.md - Interactive dashboard → FastAPI application: requests predictions
README.md - FastAPI application → Prediction endpoints: registers route
backend/app/main.py - FastAPI application → Health endpoint: registers route
backend/app/main.py - FastAPI application → Model listing: registers route
backend/app/main.py - FastAPI application → Model registry: loads models at startup
backend/app/main.py - Prediction endpoints → Inference orchestration: runs inference
backend/app/routes/predict.py - Prediction endpoints → Image loading and transforms: loads uploaded image
backend/app/routes/predict.py - Prediction endpoints → API response schemas: validates response
backend/app/routes/predict.py - Model listing → Model registry: lists model metadata
backend/app/routes/models.py - Health endpoint → Model registry: checks registry
backend/app/routes/health.py - Health endpoint → Device selection: reports device
backend/app/routes/health.py - Inference orchestration → Model registry: gets model
backend/app/inference.py - Inference orchestration → Image loading and transforms: preprocesses image
backend/app/inference.py - Inference orchestration → Device selection: selects device
backend/app/inference.py - Inference orchestration → Model configuration: uses class names
backend/app/inference.py - Model registry → Model factory: creates models
backend/app/model_registry.py - Model factory → Model configuration: reads model settings
backend/app/model_factory.py - Model factory → DenseNet121: constructs model
backend/app/model_factory.py - Model factory → MedSigLIP: constructs model
backend/app/model_factory.py - Model factory → MedSigLIP LoRA: constructs model
backend/app/model_factory.py - Model factory → ViT hybrid fusion: constructs model
backend/app/model_factory.py - MedSigLIP → Checkpoint loading: loads checkpoint
backend/app/models/medsiglip.py - MedSigLIP → Device selection: selects device
backend/app/models/medsiglip.py - MedSigLIP LoRA → Checkpoint loading: loads checkpoint
backend/app/models/medsiglip_lora.py - MedSigLIP LoRA → Device selection: selects device
backend/app/models/medsiglip_lora.py - Researcher → Model training: trains models
- Researcher → Model evaluation: evaluates models
- Researcher → Grad-CAM explainability: generates explanations
README.md - Interactive dashboard → Grad-CAM explainability: displays overlays
README.md