How vios-s/gmt-leaf-ins-seg works
Project type and purpose
• This is an ML research codebase (“gmt-leaf-ins-seg”) implementing “GMT: Guided Mask Transformer” for leaf instance segmentation. It’s built on top of Facebook’s Mask2Former and a Harmonic Embeddings library. The main use cases are training and evaluating guided mask transformer models on various plant datasets (CVPPP, Komatsuna, MSU PID).
File structure
• Top-level entries:
• configs/ ← YAML files controlling experiments per dataset
• harmonic/ ← code for generating and loading guide embeddings
• mask2former/ ← core model architectures, data loaders, criterion, decoders
• guide_train_net.py ← training orchestration script
• submission/ ← serialized results for paper
• tools/ ← helper scripts (conversion, evaluation)
• utils/ ← dataset registrations, evaluation metrics
• Patterns:
• “mask2former” follows a modular meta-architecture: backbone → pixel_decoder → transformer_decoder → head
• “harmonic” is a separate module that preprocesses or augments input with learned embeddings
• Config-driven: every experiment is wired up via YAML in configs/
• Scripts in tools/ and utils/ provide dataset registration, model conversion, evaluation
README for additional insights
• Models live in mask2former/
• Guide functions (embeddings) generated via harmonic/get_embeddings.py and stored in harmonic/guide_functions/
• Training entrypoint is guide_train_net.py
• Dependencies are managed by environment.yml (PyTorch, detectron2, etc.)