How jatin-mehra119/csiro-image2biomass works
1. Repository purpose and top-level split
This repo is organized around a single ML product: predicting pasture biomass from field images plus tabular/contextual data. Architecturally it has two main planes:
- an offline training/analysis plane at the repository root
- an online inference plane under API/
2. Inputs and data boundary
The training pipeline consumes:
- image datasets in train/ and test/
- tabular data from train.csv and test.csv
- likely derived or auxiliary features such as NDVI and ground-truth measurements, as described in the README
These inputs are the source of truth for model fitting and evaluation.
3. Offline training orchestration
The root scripts main.py and train.py are the primary training entry points. They appear to orchestrate model fitting, validation, and checkpoint creation. This is where the repo’s predictive models are built, rather than served.
4. Auxiliary modeling path
aux_model.py suggests a separate branch of modeling for auxiliary data or a secondary learner. The README explicitly references an “auxiliary model” and an “ensemble,” so the training architecture likely combines:
- a main biomass model
- one or more auxiliary models
- an ensemble or checkpoint selection strategy
5. Exploration and dataset inspection
EDA.py is a standalone analysis workflow for inspecting distributions, sample images, and summary statistics. It is not part of the serving path, but it supports model development by validating the data before training.
6. Model artifact layer
The README references saved weight directories such as Models_Seeded_Ensemble/ and Models_Aux_Only/. These are the persistence boundary between training and inference:
- training scripts write .pth model weights here
- the API loads those artifacts for prediction
7. Inference service architecture
API/ is a separate service package, likely built with FastAPI as indicated by the README. Its structure is cleanly divided into:
- API/main.py: application bootstrap / server entry point
- API/api/endpoints.py: request routes
- API/models/: model architecture and inference logic
- API/core/config.py: runtime configuration
8. Request/response contracts
API/models/schemas.py defines the API’s input and output shapes, which is important for diagramming the service boundary. This layer is what decouples client requests from internal model code and ensures stable prediction payloads.
9. Serving-time model logic
API/models/architectures.py and API/models/inference.py separate network definition from prediction execution. In the architecture diagram, this is the point where incoming API requests are transformed into model-ready inputs, then passed through loaded checkpoints to produce biomass predictions.
10. Deployment and runtime
The repo is designed to run both locally and in containerized form:
- uvicorn API.main:app --reload starts the inference server
- dockerfile packages the application for consistent deployment
The README also notes a Hugging Face Space deployment, so the same inference service is intended to be hosted remotely as an API endpoint.
11. Experiment tracking
Training is tracked in Weights & Biases, which functions as an observability layer for the offline pipeline. It records model metrics, likely across the main and auxiliary training runs, but does not affect runtime inference.
12. How the data flows end-to-end
The overall lifecycle is:
- raw image/tabular data enter the training scripts
- models are trained and evaluated offline
- best checkpoints are saved as .pth artifacts
- the API loads those checkpoints on startup
- client requests hit the endpoint layer
- inference code returns biomass predictions
13. Key architectural boundary to show in a diagram
The most important boundary is between training-time computation and serving-time inference. Everything under the repo root primarily produces model artifacts; everything under API/ consumes those artifacts and exposes them as a prediction service.
Main entry points: train.py Train loop · inference.py Inference · main.py API app · endpoints.py Routes
Components 18
Offline training
Root-level scripts and analysis used to build biomass models and checkpoints.
- Main train · Primary offline training entry point.
main.py - Train loop · Model fitting, validation, and checkpoint workflow.
train.py - Aux model · Secondary modeling path for auxiliary data or ensemble support.
aux_model.py - EDA · Dataset inspection and exploratory analysis.
EDA.py - W&B · Tracks offline training metrics and runs.
Inference API
FastAPI service that loads trained artifacts and returns predictions.
- API app · Server startup and application wiring.
API/main.py - Routes · HTTP endpoint definitions.
API/api/endpoints.py - Config · Service configuration and environment settings.
API/core/config.py - Schemas · Request and response payload definitions.
API/models/schemas.py - Arch defs · Serving-time network and model structure definitions.
API/models/architectures.py - Inference · Loads checkpoints and produces biomass predictions.
API/models/inference.py
Data and artifacts
Training inputs, generated model weights, and analysis outputs.
- Train set · Image and tabular training inputs.
- Test set · Held-out evaluation inputs.
- Train CSV · Training metadata and labels.
- Test CSV · Evaluation metadata and labels.
- Plots · Generated visualizations from exploration or training.
plots - Model weights · Saved checkpoints consumed by serving.
- Aux weights · Auxiliary model checkpoints consumed by serving.
Connections 21
- Train set → Main train: inputs · Image training data feeds the main training entry point.
- Test set → Train loop: eval data · Held-out data is used for validation and evaluation.
- Train CSV → Main train: labels · Tabular labels and metadata drive training.
- Test CSV → Train loop: eval metadata · Tabular evaluation data supports model assessment.
- EDA → Train set: inspect · Exploration validates data distributions and samples.
- EDA → Plots: outputs · EDA produces visual summaries and charts.
- Main train → Train loop: orchestrates · Primary entry point hands control to the training loop.
- Train loop → Aux model: branch · Training may invoke auxiliary modeling or ensemble paths.
- Train loop → W&B: logs · Training metrics and runs are tracked externally.
- Aux model → Aux weights: writes · Auxiliary training saves its own checkpoints.
- Train loop → Model weights: writes · Main training saves model artifacts for later serving.
- Model weights → API app: loads · API startup loads the primary trained checkpoint.
- Aux weights → Inference: loads · Inference may use auxiliary checkpoints or ensemble members.
- API app → Config: configures · Service bootstrapping reads runtime settings.
- API app → Routes: mounts · Application bootstrap attaches request routes.
- Routes → Schemas: validates · Routes use schemas to validate request and response shapes.
- Routes → Inference: calls · HTTP handlers delegate prediction work to inference logic.
- Inference → Arch defs: uses · Prediction code relies on model definitions.
- Inference → Model weights: runs · Inference executes loaded trained weights.
- Inference → Schemas: formats · Inference output is shaped for API contracts.
- API app → Inference: initializes · Server startup prepares the prediction pipeline.