How sahahahil/captioning-image works
This repository implements image captioning: training prepares image-caption examples and vocabulary, then learns a ResNet50 image encoder and LSTM caption decoder; inference uses the models and vocabulary to generate text. The README documents training and inference pipelines, but the tree contains notebooks rather than the described Python scripts, and their contents were not sampled. Pipeline links below follow documented behavior; internal notebook wiring is therefore not asserted.
Main entry points: inference.ipynb Generate caption · train.ipynb Train model · dataset.py Flickr dataset · vocab.py Vocabulary
Components 11
Data preparation
- Flickr dataset
dataset.py - Vocabulary
vocab.py - BLEU scoring
utils.py
Caption models
- ResNet encoder
model.py - LSTM decoder
model.py
User workflows
- Train model
train.ipynb - Generate caption
inference.ipynb
Other components
- Images and captions
- Input image
- User
- Generated caption
Connections 12
- Images and captions → Flickr dataset: loads examples
- Flickr dataset → Vocabulary: numericalizes captions
- Train model → Flickr dataset: trains from
- Train model → Vocabulary: builds vocabulary
- Train model → ResNet encoder: trains
- Train model → LSTM decoder: trains
- User → Generate caption: requests caption
- Input image → Generate caption: provides image
- Generate caption → ResNet encoder: encodes image
- Generate caption → LSTM decoder: decodes caption
- Generate caption → Vocabulary: uses vocabulary
- Generate caption → Generated caption: returns caption