# abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k architecture

> Architecture diagram of the GitHub repository abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-11-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k
- Repository: https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k

## Overview

This repository is a Flask image-captioning application alongside experiments comparing captioning architectures on Flickr30k. The user workflow is image upload, model selection, and caption generation; the application supports VGG19-LSTM, InceptionV3-BiLSTM, and BLIP. The notebooks document additional training experiments, including InceptionV3-Transformer, and the README reports BLEU evaluation. The sampled app excerpts show model loading and decoding implementations, but omit parts of the upload route, so some application connections rely on README documentation.

Main entry points: [`app.py`](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py) (Flask routes), [`README.md`](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/README.md) (BLEU evaluation), [`data/README.md`](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/data/README.md) (Flickr30k dataset), [`notebooks/01_cnn_bilstm_inceptionv3.ipynb`](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/notebooks/01_cnn_bilstm_inceptionv3.ipynb) (BiLSTM experiment)

## Diagram (Mermaid)

```mermaid
flowchart TD

subgraph group_web["Web application"]
  node_flask["Flask routes<br/>[app.py]"]
  node_home["Home page<br/>[home.html]"]
  node_about["About page<br/>[about.html]"]
  node_uploads[("Uploaded images")]
end

subgraph group_caption["Caption inference"]
  node_inception["Inception-BiLSTM<br/>[app.py]"]
  node_inception_features["Inception features<br/>[app.py]"]
  node_inception_decode["BiLSTM decoding<br/>[app.py]"]
  node_vgg["VGG19-LSTM<br/>[app.py]"]
  node_vgg_features["VGG19 features<br/>[app.py]"]
  node_vgg_decode["LSTM decoding<br/>[app.py]"]
  node_blip["BLIP captioning<br/>[app.py]"]
end

subgraph group_research["Model experiments"]
  node_bilstm_notebook["BiLSTM experiment"]
  node_vgg_notebook["VGG-LSTM experiment<br/>[02_vgg_lstm.ipynb]"]
  node_transformer["Transformer experiment"]
end

subgraph group_evaluation["Dataset evaluation"]
  node_flickr["Flickr30k dataset<br/>[README.md]"]
  node_bleu["BLEU evaluation<br/>[README.md]"]
end

node_user(("Image user"))
node_huggingface["Hugging Face"]

node_user -->|"uploads image"| node_flask
node_flask -->|"renders"| node_home
node_flask -->|"renders"| node_about
node_flask -->|"saves image"| node_uploads
node_flask -->|"dispatches to"| node_inception
node_flask -->|"dispatches to"| node_vgg
node_flask -->|"dispatches to"| node_blip
node_inception -->|"extracts features"| node_inception_features
node_inception -->|"generates caption"| node_inception_decode
node_vgg -->|"extracts features"| node_vgg_features
node_vgg -->|"generates caption"| node_vgg_decode
node_blip -->|"loads pretrained weights"| node_huggingface
node_flickr -.->|"trains on"| node_bilstm_notebook
node_flickr -.->|"trains on"| node_vgg_notebook
node_flickr -.->|"evaluates on"| node_transformer
node_bilstm_notebook -.->|"compares captions"| node_bleu
node_vgg_notebook -.->|"compares captions"| node_bleu
node_transformer -.->|"compares captions"| node_bleu
node_blip -.->|"reported evaluation"| node_bleu

click node_flask "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py"
click node_home "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/templates/home.html"
click node_about "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/templates/about.html"
click node_inception "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py"
click node_inception_features "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py"
click node_inception_decode "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py"
click node_vgg "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py"
click node_vgg_features "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py"
click node_vgg_decode "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py"
click node_blip "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py"
click node_bilstm_notebook "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/notebooks/01_cnn_bilstm_inceptionv3.ipynb"
click node_vgg_notebook "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/notebooks/02_vgg_lstm.ipynb"
click node_transformer "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/notebooks/01_cnn_bilstm_inceptionv3.ipynb"
click node_flickr "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/data/README.md"
click node_bleu "https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/README.md"

classDef toneNeutral fill:#f8fafc,stroke:#334155,stroke-width:1.5px,color:#0f172a
classDef toneBlue fill:#dbeafe,stroke:#2563eb,stroke-width:1.5px,color:#172554
classDef toneAmber fill:#fef3c7,stroke:#d97706,stroke-width:1.5px,color:#78350f
classDef toneMint fill:#dcfce7,stroke:#16a34a,stroke-width:1.5px,color:#14532d
classDef toneRose fill:#ffe4e6,stroke:#e11d48,stroke-width:1.5px,color:#881337
classDef toneIndigo fill:#e0e7ff,stroke:#4f46e5,stroke-width:1.5px,color:#312e81
classDef toneTeal fill:#ccfbf1,stroke:#0f766e,stroke-width:1.5px,color:#134e4a
class node_flask,node_home,node_about,node_uploads,node_user toneBlue
class node_inception,node_inception_features,node_inception_decode,node_vgg,node_vgg_features,node_vgg_decode,node_blip toneAmber
class node_bilstm_notebook,node_vgg_notebook,node_transformer toneMint
class node_flickr,node_bleu toneRose
class node_huggingface toneIndigo
```

## Components

### Web application

- [Flask routes](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py): `app.py`
- [Home page](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/templates/home.html): `templates/home.html`
- [About page](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/templates/about.html): `templates/about.html`
- **Uploaded images**

### Caption inference

- [Inception-BiLSTM](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py): `app.py`
- [Inception features](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py): `app.py`
- [BiLSTM decoding](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py): `app.py`
- [VGG19-LSTM](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py): `app.py`
- [VGG19 features](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py): `app.py`
- [LSTM decoding](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py): `app.py`
- [BLIP captioning](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/app.py): `app.py`

### Model experiments

- [BiLSTM experiment](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/notebooks/01_cnn_bilstm_inceptionv3.ipynb): `notebooks/01_cnn_bilstm_inceptionv3.ipynb`
- [VGG-LSTM experiment](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/notebooks/02_vgg_lstm.ipynb): `notebooks/02_vgg_lstm.ipynb`
- [Transformer experiment](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/notebooks/01_cnn_bilstm_inceptionv3.ipynb): `notebooks/01_cnn_bilstm_inceptionv3.ipynb`

### Dataset evaluation

- [Flickr30k dataset](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/data/README.md): `data/README.md`
- [BLEU evaluation](https://github.com/abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k/blob/main/README.md): `README.md`

### Other components

- **Image user**
- **Hugging Face**

## Connections

- Image user → Flask routes: uploads image (evidence: `README.md`)
- Flask routes → Home page: renders (evidence: `app.py`)
- Flask routes → About page: renders (evidence: `app.py`)
- Flask routes → Uploaded images: saves image (evidence: `README.md`)
- Flask routes → Inception-BiLSTM: dispatches to (evidence: `README.md`)
- Flask routes → VGG19-LSTM: dispatches to (evidence: `README.md`)
- Flask routes → BLIP captioning: dispatches to (evidence: `README.md`)
- Inception-BiLSTM → Inception features: extracts features (evidence: `app.py`)
- Inception-BiLSTM → BiLSTM decoding: generates caption (evidence: `README.md`)
- VGG19-LSTM → VGG19 features: extracts features (evidence: `app.py`)
- VGG19-LSTM → LSTM decoding: generates caption (evidence: `README.md`)
- BLIP captioning → Hugging Face: loads pretrained weights (evidence: `app.py`)
- Flickr30k dataset → BiLSTM experiment: trains on (evidence: `README.md`)
- Flickr30k dataset → VGG-LSTM experiment: trains on (evidence: `README.md`)
- Flickr30k dataset → Transformer experiment: evaluates on (evidence: `README.md`)
- BiLSTM experiment → BLEU evaluation: compares captions (evidence: `README.md`)
- VGG-LSTM experiment → BLEU evaluation: compares captions (evidence: `README.md`)
- Transformer experiment → BLEU evaluation: compares captions (evidence: `README.md`)
- BLIP captioning → BLEU evaluation: reported evaluation (evidence: `README.md`)
