# sahahahil/captioning-image architecture

> Architecture diagram of the GitHub repository sahahahil/captioning-image, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-09-29). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sahahahil/captioning-image
- Repository: https://github.com/sahahahil/captioning-image

## Overview

This repository implements image captioning: training prepares image-caption examples and vocabulary, then learns a ResNet50 image encoder and LSTM caption decoder; inference uses the models and vocabulary to generate text. The README documents training and inference pipelines, but the tree contains notebooks rather than the described Python scripts, and their contents were not sampled. Pipeline links below follow documented behavior; internal notebook wiring is therefore not asserted.

Main entry points: [`inference.ipynb`](https://github.com/sahahahil/captioning-image/blob/main/inference.ipynb) (Generate caption), [`train.ipynb`](https://github.com/sahahahil/captioning-image/blob/main/train.ipynb) (Train model), [`dataset.py`](https://github.com/sahahahil/captioning-image/blob/main/dataset.py) (Flickr dataset), [`vocab.py`](https://github.com/sahahahil/captioning-image/blob/main/vocab.py) (Vocabulary)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_data["Data preparation"]
  node_dataset["Flickr dataset<br/>[dataset.py]"]
  node_vocab["Vocabulary<br/>[vocab.py]"]
  node_bleu["BLEU scoring<br/>[utils.py]"]
end

subgraph group_models["Caption models"]
  node_encoder["ResNet encoder<br/>[model.py]"]
  node_decoder["LSTM decoder<br/>[model.py]"]
end

subgraph group_workflows["User workflows"]
  node_trainer["Train model<br/>[train.ipynb]"]
  node_infer["Generate caption<br/>[inference.ipynb]"]
end

node_training_data[("Images and captions")]
node_image["Input image"]
node_user(("User"))
node_caption["Generated caption"]

node_training_data -->|"loads examples"| node_dataset
node_dataset -->|"numericalizes captions"| node_vocab
node_trainer -->|"trains from"| node_dataset
node_trainer -->|"builds vocabulary"| node_vocab
node_trainer -->|"trains"| node_encoder
node_trainer -->|"trains"| node_decoder
node_user -->|"requests caption"| node_infer
node_image -->|"provides image"| node_infer
node_infer -->|"encodes image"| node_encoder
node_infer -->|"decodes caption"| node_decoder
node_infer -->|"uses vocabulary"| node_vocab
node_infer -->|"returns caption"| node_caption

click node_trainer "https://github.com/sahahahil/captioning-image/blob/main/train.ipynb"
click node_infer "https://github.com/sahahahil/captioning-image/blob/main/inference.ipynb"
click node_dataset "https://github.com/sahahahil/captioning-image/blob/main/dataset.py"
click node_vocab "https://github.com/sahahahil/captioning-image/blob/main/vocab.py"
click node_bleu "https://github.com/sahahahil/captioning-image/blob/main/utils.py"
click node_encoder "https://github.com/sahahahil/captioning-image/blob/main/model.py"
click node_decoder "https://github.com/sahahahil/captioning-image/blob/main/model.py"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_dataset,node_vocab,node_bleu,node_user toneBlue
class node_encoder,node_decoder,node_training_data toneAmber
class node_trainer,node_infer toneMint
class node_image,node_caption toneIndigo
```

## Components

### Data preparation

- [Flickr dataset](https://github.com/sahahahil/captioning-image/blob/main/dataset.py): `dataset.py`
- [Vocabulary](https://github.com/sahahahil/captioning-image/blob/main/vocab.py): `vocab.py`
- [BLEU scoring](https://github.com/sahahahil/captioning-image/blob/main/utils.py): `utils.py`

### Caption models

- [ResNet encoder](https://github.com/sahahahil/captioning-image/blob/main/model.py): `model.py`
- [LSTM decoder](https://github.com/sahahahil/captioning-image/blob/main/model.py): `model.py`

### User workflows

- [Train model](https://github.com/sahahahil/captioning-image/blob/main/train.ipynb): `train.ipynb`
- [Generate caption](https://github.com/sahahahil/captioning-image/blob/main/inference.ipynb): `inference.ipynb`

### Other components

- **Images and captions**
- **Input image**
- **User**
- **Generated caption**

## Connections

- Images and captions → Flickr dataset: loads examples
- Flickr dataset → Vocabulary: numericalizes captions
- Train model → Flickr dataset: trains from
- Train model → Vocabulary: builds vocabulary
- Train model → ResNet encoder: trains
- Train model → LSTM decoder: trains
- User → Generate caption: requests caption
- Input image → Generate caption: provides image
- Generate caption → ResNet encoder: encodes image
- Generate caption → LSTM decoder: decodes caption
- Generate caption → Vocabulary: uses vocabulary
- Generate caption → Generated caption: returns caption