How cynapticsai/isro_interiit_14 works
This repository is a remote-sensing multimodal model research workflow: it prepares caption, VQA, and grounding data; fine-tunes Qwen2.5-VL; runs inference; and scores or benchmarks generated results. The sampled code supports image indexing and SAR/IR data preparation, GPT-assisted synthetic caption/VQA generation, grounding conversion, task classification, model inference, and evaluation. Benchmark scripts and several fine-tuning entry points are visible in the tree but were not sampled, so their internal wiring is not inferred.
Main entry points: multimodal_adapters LoRA adapters · convert_detection.py Detection conversion · inference_hybrid.py Hybrid inference · results Inference results
Components 20
Data preparation
- Detection conversion
dataset/convert_detection.py - Grounding conversion
dataset/convert_grounding.py - Task classification
other/classify_json.py - Task CSV export
other/convert_skysense.py
Model adaptation
- SAR/IR preparation
fine_tuning/multimodality_pipeline.py - Caption training
fine_tuning/finetune_caption.py - VQA training
fine_tuning/finetune_vqa.py - Hybrid training
fine_tuning/finetune_hybrid.py - LoRA adapters
fine_tuning/multimodal_adapters
Inference
- Caption inference
fine_tuning/inference_caption.py - VQA inference
fine_tuning/inference_vqa.py - Hybrid inference
fine_tuning/inference_hybrid.py - Inference results
fine_tuning/results
Evaluation
- Fine-tune evaluation
fine_tuning/evaluate.py - Model benchmark
benchmark/scripts/evaluate.py - Benchmark results
benchmark/results
Other components
- Researcher
- MMRS-1M data
- SkySense data
- GPT-4o
Connections 31
- Researcher → Detection conversion: runs generation
- MMRS-1M data → Detection conversion: provides annotations
- Detection conversion → GPT-4o: requests captions/Q&A
- Detection conversion → Grounding conversion: shares detection data
- MMRS-1M data → Grounding conversion: provides annotations
- Researcher → Task classification: runs classification
- SkySense data → Task classification: provides task records
- Task classification → Task CSV export: classified JSON
- Researcher → SAR/IR preparation: starts training
- MMRS-1M data → SAR/IR preparation: provides images
- SAR/IR preparation → LoRA adapters: saves adapters
- Researcher → Caption training: starts training
- Researcher → VQA training: starts training
- Researcher → Hybrid training: starts training
- Caption training → LoRA adapters: saves adapter
- VQA training → LoRA adapters: saves adapter
- Hybrid training → LoRA adapters: saves adapter
- Researcher → Caption inference: runs inference
- Researcher → VQA inference: runs inference
- Researcher → Hybrid inference: runs inference
- LoRA adapters → Caption inference: loads adapter
- LoRA adapters → VQA inference: loads adapter
- LoRA adapters → Hybrid inference: loads adapter
- SkySense data → Hybrid inference: provides evaluation data
- Caption inference → Inference results: writes predictions
- VQA inference → Inference results: writes predictions
- Hybrid inference → Inference results: writes predictions
- Researcher → Fine-tune evaluation: runs scoring
- Inference results → Fine-tune evaluation: reads predictions
- Researcher → Model benchmark: runs benchmark
- Model benchmark → Benchmark results: writes scores