# aarohisingla/image-captioning architecture

> Architecture diagram of the GitHub repository aarohisingla/image-captioning, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-25). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/aarohisingla/image-captioning
- Repository: https://github.com/aarohisingla/image-captioning

## Overview

This repository implements CNN–RNN image captioning: COCO images and captions are prepared as model inputs, an image encoder and recurrent decoder produce captions, and NLP utilities clean predictions and calculate BLEU. The README identifies training and inference notebooks, but their contents were not sampled, so notebook-to-module wiring is not asserted. The loader, dataset, vocabulary, model, and evaluation relationships shown are supported by sampled source or README descriptions.

Main entry points: [`vocabulary.py`](https://github.com/aarohisingla/image-captioning/blob/main/vocabulary.py) (Vocabulary), [`data_loader.py`](https://github.com/aarohisingla/image-captioning/blob/main/data_loader.py) (Training loader), [`data_loader_val.py`](https://github.com/aarohisingla/image-captioning/blob/main/data_loader_val.py) (Validation loader), [`Inference.ipynb`](https://github.com/aarohisingla/image-captioning/blob/main/Inference.ipynb) (Inference workflow)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_train_loader["Training loader<br/>[data_loader.py]"]
  node_val_loader["Validation loader<br/>[data_loader_val.py]"]
  node_dataset["COCO dataset reader<br/>[coco_dataset.py]"]
  node_val_dataset["Validation dataset reader<br/>[data_loader_val.py]"]
  node_vocabulary["Vocabulary<br/>[vocabulary.py]"]
end

subgraph group_model["Caption model"]
  node_encoder["CNN encoder<br/>[model.py]"]
  node_decoder["RNN decoder<br/>[model.py]"]
end

subgraph group_workflow["Training and inference"]
  node_training["Training workflow<br/>[Training.ipynb]"]
  node_inference["Inference workflow<br/>[Inference.ipynb]"]
end

subgraph group_evaluation["Caption evaluation"]
  node_clean["Caption cleanup<br/>[nlp_utils.py]"]
  node_bleu["BLEU scoring<br/>[nlp_utils.py]"]
end

node_user(("Engineer"))
node_coco[("COCO dataset")]
node_vocab_file[("Vocabulary file<br/>[vocab.pkl]")]

node_train_loader -->|"constructs"| node_dataset
node_train_loader -->|"selects files"| node_coco
node_val_loader -->|"constructs"| node_val_dataset
node_val_loader -->|"selects files"| node_coco
node_dataset -->|"constructs"| node_vocabulary
node_val_dataset -->|"constructs"| node_vocabulary
node_vocabulary -->|"loads or writes"| node_vocab_file
node_vocabulary -->|"reads captions"| node_coco
node_encoder -->|"features"| node_decoder
node_training -->|"initiated by"| node_user
node_inference -->|"initiated by"| node_user
node_inference -->|"cleans captions"| node_clean
node_bleu --> node_clean

click node_train_loader "https://github.com/aarohisingla/image-captioning/blob/main/data_loader.py"
click node_val_loader "https://github.com/aarohisingla/image-captioning/blob/main/data_loader_val.py"
click node_dataset "https://github.com/aarohisingla/image-captioning/blob/main/coco_dataset.py"
click node_val_dataset "https://github.com/aarohisingla/image-captioning/blob/main/data_loader_val.py"
click node_vocabulary "https://github.com/aarohisingla/image-captioning/blob/main/vocabulary.py"
click node_vocab_file "https://github.com/aarohisingla/image-captioning/blob/main/vocab.pkl"
click node_encoder "https://github.com/aarohisingla/image-captioning/blob/main/model.py"
click node_decoder "https://github.com/aarohisingla/image-captioning/blob/main/model.py"
click node_training "https://github.com/aarohisingla/image-captioning/blob/main/Training.ipynb"
click node_inference "https://github.com/aarohisingla/image-captioning/blob/main/Inference.ipynb"
click node_clean "https://github.com/aarohisingla/image-captioning/blob/main/nlp_utils.py"
click node_bleu "https://github.com/aarohisingla/image-captioning/blob/main/nlp_utils.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_train_loader,node_val_loader,node_dataset,node_val_dataset,node_vocabulary toneBlue
class node_encoder,node_decoder,node_coco,node_vocab_file toneAmber
class node_training,node_inference toneMint
class node_clean,node_bleu toneRose
class node_user toneIndigo
```

## Components

### Data preparation

- [Training loader](https://github.com/aarohisingla/image-captioning/blob/main/data_loader.py): `data_loader.py`
- [Validation loader](https://github.com/aarohisingla/image-captioning/blob/main/data_loader_val.py): `data_loader_val.py`
- [COCO dataset reader](https://github.com/aarohisingla/image-captioning/blob/main/coco_dataset.py): `coco_dataset.py`
- [Validation dataset reader](https://github.com/aarohisingla/image-captioning/blob/main/data_loader_val.py): `data_loader_val.py`
- [Vocabulary](https://github.com/aarohisingla/image-captioning/blob/main/vocabulary.py): `vocabulary.py`

### Caption model

- [CNN encoder](https://github.com/aarohisingla/image-captioning/blob/main/model.py): `model.py`
- [RNN decoder](https://github.com/aarohisingla/image-captioning/blob/main/model.py): `model.py`

### Training and inference

- [Training workflow](https://github.com/aarohisingla/image-captioning/blob/main/Training.ipynb): `Training.ipynb`
- [Inference workflow](https://github.com/aarohisingla/image-captioning/blob/main/Inference.ipynb): `Inference.ipynb`

### Caption evaluation

- [Caption cleanup](https://github.com/aarohisingla/image-captioning/blob/main/nlp_utils.py): `nlp_utils.py`
- [BLEU scoring](https://github.com/aarohisingla/image-captioning/blob/main/nlp_utils.py): `nlp_utils.py`

### Other components

- **Engineer**
- **COCO dataset**
- [Vocabulary file](https://github.com/aarohisingla/image-captioning/blob/main/vocab.pkl): `vocab.pkl`

## Connections

- Training loader → COCO dataset reader: constructs
- Training loader → COCO dataset: selects files
- Validation loader → Validation dataset reader: constructs
- Validation loader → COCO dataset: selects files
- COCO dataset reader → Vocabulary: constructs
- Validation dataset reader → Vocabulary: constructs
- Vocabulary → Vocabulary file: loads or writes
- Vocabulary → COCO dataset: reads captions
- CNN encoder → RNN decoder: features
- Training workflow → Engineer: initiated by
- Inference workflow → Engineer: initiated by
- Inference workflow → Caption cleanup: cleans captions
- BLEU scoring → Caption cleanup