How aarohisingla/image-captioning works
This repository implements CNN–RNN image captioning: COCO images and captions are prepared as model inputs, an image encoder and recurrent decoder produce captions, and NLP utilities clean predictions and calculate BLEU. The README identifies training and inference notebooks, but their contents were not sampled, so notebook-to-module wiring is not asserted. The loader, dataset, vocabulary, model, and evaluation relationships shown are supported by sampled source or README descriptions.
Main entry points: vocabulary.py Vocabulary · data_loader.py Training loader · data_loader_val.py Validation loader · Inference.ipynb Inference workflow
Components 14
Data preparation
- Training loader
data_loader.py - Validation loader
data_loader_val.py - COCO dataset reader
coco_dataset.py - Validation dataset reader
data_loader_val.py - Vocabulary
vocabulary.py
Caption model
- CNN encoder
model.py - RNN decoder
model.py
Training and inference
- Training workflow
Training.ipynb - Inference workflow
Inference.ipynb
Caption evaluation
- Caption cleanup
nlp_utils.py - BLEU scoring
nlp_utils.py
Other components
- Engineer
- COCO dataset
- Vocabulary file
vocab.pkl
Connections 13
- Training loader → COCO dataset reader: constructs
- Training loader → COCO dataset: selects files
- Validation loader → Validation dataset reader: constructs
- Validation loader → COCO dataset: selects files
- COCO dataset reader → Vocabulary: constructs
- Validation dataset reader → Vocabulary: constructs
- Vocabulary → Vocabulary file: loads or writes
- Vocabulary → COCO dataset: reads captions
- CNN encoder → RNN decoder: features
- Training workflow → Engineer: initiated by
- Inference workflow → Engineer: initiated by
- Inference workflow → Caption cleanup: cleans captions
- BLEU scoring → Caption cleanup