How rogerjaden/deep-hybrid-model-for-classification-of-femoral-stem works
This repository is a research-grade offline ML pipeline for 3-class image classification (hip X-ray femoral stem type) with:
• Training (main.py)
• Evaluation + metrics/plots (evaluate.py, final_metrics_plot.py)
• Inference + explainability (Grad-CAM) (predict.py, gradcam.py)
• Model package (models/) implementing MSFT-Net (ResNet backbone + multi-scale features + ECA attention + Transformer encoder + classifier head)
• Shared utilities (utils/) for dataset loading, transforms, training loop, and metrics
So the “system” here is not a web service; it’s a batch/offline ML workflow that consumes image folders, produces model artifacts + reports, and optionally produces Grad-CAM visual explanations.
From the tree:
Entry-point pipelines (top-level scripts)
• main.py — training pipeline (dataset → model → optimizer → checkpoints/results)
• evaluate.py — evaluation pipeline (load weights → run test set → compute metrics → write results)
• predict.py — inference pipeline (single/few images → prediction → Grad-CAM outputs)
• final_metrics_plot.py — reporting/plotting pipeline (reads results/metrics.json → produces plots)
• gradcam.py — explainability engine used by prediction (and possibly evaluation)
Core libraries (imported by entrypoints)
These are internal modules; show them as shared components:
• utils/dataset.py — dataset + dataloader wiring (reads directory-structured dataset)
• utils/transforms.py — preprocessing/augmentation (Albumentations/torchvision style)
• utils/engine.py — training/validation loops, device mgmt, epoch iteration
• utils/metrics.py — F1, ROC-AUC, confusion matrix (scikit-learn style)
Model implementation
• models/msftnet.py — MSFT-Net definition (the central “compute” component)
• models/eca.py — ECA attention module (subcomponent used inside MSFT-Net)
Artifacts / outputs
The repo strongly implies a local artifact store:
• results/metrics.json (canonical numeric metrics artifact)
• results/evaluation/ (confusion matrix, ROC, per-class outputs)
• results/plots/ (figures such as F1 comparison, key metrics)
• results/gradcam/ (Grad-CAM overlay images)
• test_images/ (inference inputs / demos)
Training
• dataset/ (image folders) → utils/transforms.py → utils/dataset.py → main.py
• main.py → (trained weights/checkpoints) → results/ (and/or a weights file path)
• Optionally main.py → logs → Logs/
Evaluation
• dataset/ (test split) → utils/dataset.py → evaluate.py
• evaluate.py + (load trained weights) → metrics computations (utils/metrics.py) → write:
• results/metrics.json
• results/evaluation/* (confusion matrix, ROC, etc.)
Reporting
• results/metrics.json → final_metrics_plot.py → results/plots/*
Inference + Explainability
• test_images/ (or arbitrary input images) → predict.py → utils/transforms.py
• predict.py + (load trained weights) → model forward pass (models/msftnet.py)
• predict.py → Grad-CAM (gradcam.py) → results/gradcam/*
These describe imports/usage:
• main.py → utils/engine.py, utils/metrics.py, models/msftnet.py
• evaluate.py → utils/dataset.py, utils/metrics.py, models/msftnet.py
• predict.py → models/msftnet.py, gradcam.py, utils/transforms.py
• models/msftnet.py → models/eca.py (and backbone library: timm/torchvision)
Pipeline separation by concern
• Training / evaluation / inference are separate entrypoints, each producing distinct artifacts.
• Shared utilities reduce duplication (dataset, transforms, engine, metrics).
Hybrid model architecture
• CNN backbone (ResNet-50, pretrained via timm) for local feature extraction
• Multi-scale feature usage (C2–C5 concept from README)
• ECA attention module as lightweight channel attention
• Transformer encoder for global context
• Classifier head for 3-way output
Explainable AI as first-class output
• Grad-CAM generation is integrated as a pipeline stage, producing interpretable overlays.
File-system as the “integration bus”
• There is no DB/message queue; the integration mechanism is persisted artifacts in results/ and (implied) saved weights + logs.
View 1 — “Workflow / orchestration” (highest value)
Purpose: show how a user runs the system and what gets produced.
• Inputs: dataset/, test_images/
• Runners: main.py, evaluate.py, predict.py, final_metrics_plot.py
• Shared libs: utils/*, models/*, gradcam.py
• Artifacts: results/* (+ optional Logs/)
• Solid = data/artifacts (images, weights, metrics.json, plots, heatmaps)
This should closely mirror the repo’s reality.
View 2 — “Model internal architecture (MSFT-Net)”
Purpose: show what is inside models/msftnet.py at a block level.
Recommended blocks:
• Input tensor (post-transform)
• ResNet-50 backbone (pretrained)
• Multi-scale feature aggregation (C2/C3/C4/C5)
• ECA attention (from models/eca.py)
• Transformer encoder module
• Pooling/flatten + classifier head
• Softmax/logits → 3 classes
Short annotations: “local features”, “channel recalibration”, “global context”.
View 3 — “Metrics + explainability subsystem”
Purpose: separate “model performance reporting” from “prediction explainability”.
Show:
• utils/metrics.py generating F1/ROC-AUC/confusion matrix
• final_metrics_plot.py consuming results/metrics.json
• gradcam.py producing overlays stored under results/gradcam/
This is valuable because it highlights the research/clinical trust aspects.