itorority0118/image-captioning

How itorority0118/image-captioning works

This repository is an image-captioning research project comparing four image encoders and two decoder families on Flickr30k. The principal workflow prepares caption data and image splits, extracts reusable image features, trains a caption decoder, generates captions, and evaluates them. A separate documented Gradio demo is also a product capability, though its implementation is absent from the tree. The training excerpt uses precomputed features; evaluation and demo details are documented but unsampled, so their connections are left unverified.

Main entry points: train.py Training runner · app Gradio demo · dataset.py Caption dataset · text_preprocess.py Caption preprocessing

Components 19

Data preparation

Caption preprocessing src/data/text_preprocess.py
Dataset splits src/data/split.py
Image exploration src/data/explore_images.py
Caption dataset src/data/dataset.py

Image features

Image encoders src/models/encoders.py
Feature extraction src/data/image_features.py
Feature files src/paths.py

Model training

Training runner src/train.py
GPT-2 decoder src/models/decoder_gpt2.py
LSTM attention src/models/decoder_lstm.py
Checkpoints src/paths.py

Caption generation

Greedy decoding src/decoding/greedy.py
Beam decoding src/decoding/beam.py
Diverse beam src/decoding/diverse_beam.py

Evaluation and demo

Caption metrics src/eval
Gradio demo app

Other components

Researcher
Flickr30k files
Pretrained model hub

Connections 18

Researcher → Gradio demo: uses README.md
Flickr30k files → Caption dataset: captions and images README.md
Flickr30k files → Image exploration: audits src/data/explore_images.py
Caption dataset → Caption preprocessing: uses src/data/dataset.py
Feature extraction → Image encoders: extracts with src/data/image_features.py
Feature extraction → Feature files: writes features src/data/image_features.py
Training runner → Caption dataset: loads src/train.py
Training runner → Feature files: locates features src/train.py
Training runner → GPT-2 decoder: constructs src/train.py
Training runner → LSTM attention: constructs src/train.py
Training runner → Checkpoints: locates checkpoints src/train.py
Greedy decoding → Caption preprocessing: decodes tokens src/decoding/greedy.py
Beam decoding → Caption preprocessing: decodes tokens src/decoding/beam.py
Image encoders → Pretrained model hub: loads pretrained weights src/models/encoders.py
GPT-2 decoder → Pretrained model hub: loads GPT-2 src/models/decoder_gpt2.py
Training runner → Caption metrics: evaluates
Gradio demo → Greedy decoding: generates captions
Gradio demo → Beam decoding: generates captions

Drawn by GitDiagram on November 11, 2026 from the repository's file tree, README and source files. Read as Markdown ·