How srirammayan/brain-tumor-smdfnet-swimbtc works
Purpose and entry points
This repository implements a Python MRI/CT brain-tumor classification system. It supports independent MRI and CT predictions plus a learned multimodal fusion prediction. The interactive runtime entry point is app.py; model training entry points are train_classifier.py for single-modality classifiers and fusion_train.py for fusion training. inference.py likely provides reusable prediction/loading logic.
Interactive analysis application
app.py hosts the Gradio interface. Users upload MRI and CT DICOM images, trigger analysis, and receive modality-specific and fused predictions alongside visual outputs such as heatmaps, fused images, probability plots, and confidence comparisons.
The UI is the boundary between user-provided clinical image files and model inference. DICOM decoding and image preprocessing must safely handle both absent modality inputs and malformed uploads.
Single-modality classifier stage
model_classifier.py defines the MRI and CT classifier architecture: a hybrid Swin Transformer plus EfficientNet model, instantiated separately per modality. It converts a preprocessed image tensor into tumor-class logits/probabilities and may expose intermediate feature representations needed by fusion.
train_classifier.py owns the training lifecycle for these modality-specific models: dataset ingestion, optimization, validation, metric calculation, and checkpoint creation.
Fusion modeling stage
fusion_model.py defines the CNN-based multimodal fusion network. Its input boundary is MRI and CT representations or tensors; its output is a fused tumor prediction. The architecture is responsible for combining complementary modality evidence rather than simply averaging independent classifier probabilities.
fusion_train.py trains and validates this fusion model, relying on aligned MRI/CT training samples and producing weights compatible with application inference.
Inference and visualization flow
inference.py is the likely inference orchestration layer used by the app: load available trained checkpoints, preprocess MRI/CT DICOM data into model inputs, run each classifier and the fusion model, and return prediction probabilities/metadata.
The primary flow is:
1. app.py receives uploaded MRI and CT DICOM files.
2. DICOM/image preprocessing produces model-ready tensors.
3. model_classifier.py models generate MRI and CT predictions/features.
4. fusion_model.py produces a multimodal prediction when both modalities are available.
5. Results return to app.py for Gradio visualizations and validation/confidence reporting.
Runtime dependencies and model availability
The system runs on Python 3.11+ with PyTorch and Torchvision/Timm for deep learning, Gradio for the UI, PyDICOM for medical-image input, and NumPy/OpenCV/Matplotlib for image processing and charts. There is no indicated external API, database, or cloud service.
Checkpoint availability is an important operational boundary: per README.md, the application falls back to randomly initialized models when trained weights cannot be found, so predictions are only clinically meaningful when compatible artifacts from train_classifier.py and fusion_train.py are present.
Main entry points: inference.py Inference orchestration · app.py Gradio analysis UI · model_classifier.py MRI / CT classifiers · fusion_model.py Multimodal fusion network
Components 13
Interactive Runtime
- Clinical user
- Gradio analysis UI · Accepts MRI and CT uploads and renders predictions and visual outputs.
app.py - MRI / CT DICOM uploads · Either modality may be absent or malformed.
- Inference orchestration · Loads models, preprocesses DICOM inputs, and returns prediction metadata.
inference.py - DICOM preprocessing · Decodes and transforms uploads into model-ready tensors.
inference.py - Heatmaps and confidence plots
app.py
Modeling
- MRI / CT classifiers · Separate modality models emit tumor predictions and fusion features.
model_classifier.py - Multimodal fusion network · Combines MRI and CT evidence into a fused tumor prediction.
fusion_model.py - Trained model checkpoints · Missing compatible weights cause random-model fallback.
README.md
Training Pipelines
- Single-modality training · Trains, validates, and evaluates independent MRI and CT classifiers.
train_classifier.py - Modality training data
- Fusion training · Trains and validates fusion using aligned MRI/CT samples.
fusion_train.py - Aligned MRI / CT samples
Connections 18
- Clinical user → MRI / CT DICOM uploads: provides
- MRI / CT DICOM uploads → Gradio analysis UI: uploads through
- Gradio analysis UI → Inference orchestration: requests analysis
- Inference orchestration → DICOM preprocessing: decodes and prepares
- DICOM preprocessing → MRI / CT classifiers: modality tensors
- MRI / CT classifiers → Multimodal fusion network: predictions / features
- Inference orchestration → Multimodal fusion network: runs when both available
- MRI / CT classifiers → Inference orchestration: MRI / CT results
- Multimodal fusion network → Inference orchestration: fused result
- Inference orchestration → Gradio analysis UI: predictions and metadata
- Gradio analysis UI → Heatmaps and confidence plots: renders
- Trained model checkpoints → Inference orchestration: loads weights
- Modality training data → Single-modality training: trains on
- Single-modality training → MRI / CT classifiers: optimizes architecture
- Single-modality training → Trained model checkpoints: writes classifier weights
- Aligned MRI / CT samples → Fusion training: trains on
- Fusion training → Multimodal fusion network: optimizes architecture
- Fusion training → Trained model checkpoints: writes fusion weights