# cynapticsai/isro_interiit_14 architecture

> Architecture diagram of the GitHub repository cynapticsai/isro_interiit_14, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/cynapticsai/isro_interiit_14
- Repository: https://github.com/cynapticsai/isro_interiit_14

## Overview

This repository is a remote-sensing multimodal model research workflow: it prepares caption, VQA, and grounding data; fine-tunes Qwen2.5-VL; runs inference; and scores or benchmarks generated results. The sampled code supports image indexing and SAR/IR data preparation, GPT-assisted synthetic caption/VQA generation, grounding conversion, task classification, model inference, and evaluation. Benchmark scripts and several fine-tuning entry points are visible in the tree but were not sampled, so their internal wiring is not inferred.

Main entry points: [`fine_tuning/multimodal_adapters`](https://github.com/cynapticsai/isro_interiit_14/tree/main/fine_tuning/multimodal_adapters) (LoRA adapters), [`dataset/convert_detection.py`](https://github.com/cynapticsai/isro_interiit_14/blob/main/dataset/convert_detection.py) (Detection conversion), [`fine_tuning/inference_hybrid.py`](https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/inference_hybrid.py) (Hybrid inference), [`fine_tuning/results`](https://github.com/cynapticsai/isro_interiit_14/tree/main/fine_tuning/results) (Inference results)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_detection["Detection conversion"]
  node_grounding["Grounding conversion"]
  node_taskclass["Task classification<br/>[classify_json.py]"]
  node_taskcsv["Task CSV export"]
end

subgraph group_training["Model adaptation"]
  node_multimodal["SAR/IR preparation"]
  node_captiontrain["Caption training"]
  node_vqatrain["VQA training<br/>[finetune_vqa.py]"]
  node_hybridtrain["Hybrid training<br/>[finetune_hybrid.py]"]
  node_adapters[("LoRA adapters")]
end

subgraph group_inference["Inference"]
  node_captioninfer["Caption inference"]
  node_vqainfer["VQA inference<br/>[inference_vqa.py]"]
  node_hybridinfer["Hybrid inference"]
  node_results[("Inference results")]
end

subgraph group_evaluation["Evaluation"]
  node_finetuneeval["Fine-tune evaluation<br/>[evaluate.py]"]
  node_benchmark["Model benchmark<br/>[evaluate.py]"]
  node_benchmarkresults[("Benchmark results")]
end

node_researcher(("Researcher"))
node_mmrs[("MMRS-1M data")]
node_skysense[("SkySense data")]
node_gpt{{"GPT-4o"}}

node_researcher -->|"runs generation"| node_detection
node_mmrs -->|"provides annotations"| node_detection
node_detection -->|"requests captions/Q&amp;A"| node_gpt
node_detection -.->|"shares detection data"| node_grounding
node_mmrs -->|"provides annotations"| node_grounding
node_researcher -->|"runs classification"| node_taskclass
node_skysense -->|"provides task records"| node_taskclass
node_taskclass -->|"classified JSON"| node_taskcsv
node_researcher -->|"starts training"| node_multimodal
node_mmrs -->|"provides images"| node_multimodal
node_multimodal -->|"saves adapters"| node_adapters
node_researcher -->|"starts training"| node_captiontrain
node_researcher -->|"starts training"| node_vqatrain
node_researcher -->|"starts training"| node_hybridtrain
node_captiontrain -->|"saves adapter"| node_adapters
node_vqatrain -->|"saves adapter"| node_adapters
node_hybridtrain -->|"saves adapter"| node_adapters
node_researcher -->|"runs inference"| node_captioninfer
node_researcher -->|"runs inference"| node_vqainfer
node_researcher -->|"runs inference"| node_hybridinfer
node_adapters -->|"loads adapter"| node_captioninfer
node_adapters -->|"loads adapter"| node_vqainfer
node_adapters -->|"loads adapter"| node_hybridinfer
node_skysense -->|"provides evaluation data"| node_hybridinfer
node_captioninfer -->|"writes predictions"| node_results
node_vqainfer -->|"writes predictions"| node_results
node_hybridinfer -->|"writes predictions"| node_results
node_researcher -->|"runs scoring"| node_finetuneeval
node_results -->|"reads predictions"| node_finetuneeval
node_researcher -->|"runs benchmark"| node_benchmark
node_benchmark -->|"writes scores"| node_benchmarkresults

click node_detection "https://github.com/cynapticsai/isro_interiit_14/blob/main/dataset/convert_detection.py"
click node_grounding "https://github.com/cynapticsai/isro_interiit_14/blob/main/dataset/convert_grounding.py"
click node_taskclass "https://github.com/cynapticsai/isro_interiit_14/blob/main/other/classify_json.py"
click node_taskcsv "https://github.com/cynapticsai/isro_interiit_14/blob/main/other/convert_skysense.py"
click node_multimodal "https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/multimodality_pipeline.py"
click node_captiontrain "https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/finetune_caption.py"
click node_vqatrain "https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/finetune_vqa.py"
click node_hybridtrain "https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/finetune_hybrid.py"
click node_adapters "https://github.com/cynapticsai/isro_interiit_14/tree/main/fine_tuning/multimodal_adapters"
click node_captioninfer "https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/inference_caption.py"
click node_vqainfer "https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/inference_vqa.py"
click node_hybridinfer "https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/inference_hybrid.py"
click node_results "https://github.com/cynapticsai/isro_interiit_14/tree/main/fine_tuning/results"
click node_finetuneeval "https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/evaluate.py"
click node_benchmark "https://github.com/cynapticsai/isro_interiit_14/blob/main/benchmark/scripts/evaluate.py"
click node_benchmarkresults "https://github.com/cynapticsai/isro_interiit_14/tree/main/benchmark/results"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_detection,node_grounding,node_taskclass,node_taskcsv toneBlue
class node_multimodal,node_captiontrain,node_vqatrain,node_hybridtrain,node_adapters,node_mmrs,node_skysense toneAmber
class node_captioninfer,node_vqainfer,node_hybridinfer,node_results toneMint
class node_finetuneeval,node_benchmark,node_benchmarkresults toneRose
class node_researcher,node_gpt toneIndigo
```

## Components

### Data preparation

- [Detection conversion](https://github.com/cynapticsai/isro_interiit_14/blob/main/dataset/convert_detection.py): `dataset/convert_detection.py`
- [Grounding conversion](https://github.com/cynapticsai/isro_interiit_14/blob/main/dataset/convert_grounding.py): `dataset/convert_grounding.py`
- [Task classification](https://github.com/cynapticsai/isro_interiit_14/blob/main/other/classify_json.py): `other/classify_json.py`
- [Task CSV export](https://github.com/cynapticsai/isro_interiit_14/blob/main/other/convert_skysense.py): `other/convert_skysense.py`

### Model adaptation

- [SAR/IR preparation](https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/multimodality_pipeline.py): `fine_tuning/multimodality_pipeline.py`
- [Caption training](https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/finetune_caption.py): `fine_tuning/finetune_caption.py`
- [VQA training](https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/finetune_vqa.py): `fine_tuning/finetune_vqa.py`
- [Hybrid training](https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/finetune_hybrid.py): `fine_tuning/finetune_hybrid.py`
- [LoRA adapters](https://github.com/cynapticsai/isro_interiit_14/tree/main/fine_tuning/multimodal_adapters): `fine_tuning/multimodal_adapters`

### Inference

- [Caption inference](https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/inference_caption.py): `fine_tuning/inference_caption.py`
- [VQA inference](https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/inference_vqa.py): `fine_tuning/inference_vqa.py`
- [Hybrid inference](https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/inference_hybrid.py): `fine_tuning/inference_hybrid.py`
- [Inference results](https://github.com/cynapticsai/isro_interiit_14/tree/main/fine_tuning/results): `fine_tuning/results`

### Evaluation

- [Fine-tune evaluation](https://github.com/cynapticsai/isro_interiit_14/blob/main/fine_tuning/evaluate.py): `fine_tuning/evaluate.py`
- [Model benchmark](https://github.com/cynapticsai/isro_interiit_14/blob/main/benchmark/scripts/evaluate.py): `benchmark/scripts/evaluate.py`
- [Benchmark results](https://github.com/cynapticsai/isro_interiit_14/tree/main/benchmark/results): `benchmark/results`

### Other components

- **Researcher**
- **MMRS-1M data**
- **SkySense data**
- **GPT-4o**

## Connections

- Researcher → Detection conversion: runs generation
- MMRS-1M data → Detection conversion: provides annotations
- Detection conversion → GPT-4o: requests captions/Q&A
- Detection conversion → Grounding conversion: shares detection data
- MMRS-1M data → Grounding conversion: provides annotations
- Researcher → Task classification: runs classification
- SkySense data → Task classification: provides task records
- Task classification → Task CSV export: classified JSON
- Researcher → SAR/IR preparation: starts training
- MMRS-1M data → SAR/IR preparation: provides images
- SAR/IR preparation → LoRA adapters: saves adapters
- Researcher → Caption training: starts training
- Researcher → VQA training: starts training
- Researcher → Hybrid training: starts training
- Caption training → LoRA adapters: saves adapter
- VQA training → LoRA adapters: saves adapter
- Hybrid training → LoRA adapters: saves adapter
- Researcher → Caption inference: runs inference
- Researcher → VQA inference: runs inference
- Researcher → Hybrid inference: runs inference
- LoRA adapters → Caption inference: loads adapter
- LoRA adapters → VQA inference: loads adapter
- LoRA adapters → Hybrid inference: loads adapter
- SkySense data → Hybrid inference: provides evaluation data
- Caption inference → Inference results: writes predictions
- VQA inference → Inference results: writes predictions
- Hybrid inference → Inference results: writes predictions
- Researcher → Fine-tune evaluation: runs scoring
- Inference results → Fine-tune evaluation: reads predictions
- Researcher → Model benchmark: runs benchmark
- Model benchmark → Benchmark results: writes scores