How abdessamadaitbrahim/deep-learning-based-image-captioning-using-flickr30k works
This repository is a Flask image-captioning application alongside experiments comparing captioning architectures on Flickr30k. The user workflow is image upload, model selection, and caption generation; the application supports VGG19-LSTM, InceptionV3-BiLSTM, and BLIP. The notebooks document additional training experiments, including InceptionV3-Transformer, and the README reports BLEU evaluation. The sampled app excerpts show model loading and decoding implementations, but omit parts of the upload route, so some application connections rely on README documentation.
Main entry points: app.py Flask routes · README.md BLEU evaluation · data/README.md Flickr30k dataset · 01_cnn_bilstm_inceptionv3.ipynb BiLSTM experiment
Components 18
Web application
- Flask routes
app.py - Home page
templates/home.html - About page
templates/about.html - Uploaded images
Caption inference
- Inception-BiLSTM
app.py - Inception features
app.py - BiLSTM decoding
app.py - VGG19-LSTM
app.py - VGG19 features
app.py - LSTM decoding
app.py - BLIP captioning
app.py
Model experiments
- BiLSTM experiment
notebooks/01_cnn_bilstm_inceptionv3.ipynb - VGG-LSTM experiment
notebooks/02_vgg_lstm.ipynb - Transformer experiment
notebooks/01_cnn_bilstm_inceptionv3.ipynb
Dataset evaluation
- Flickr30k dataset
data/README.md - BLEU evaluation
README.md
Other components
- Image user
- Hugging Face
Connections 19
- Image user → Flask routes: uploads image
README.md - Flask routes → Home page: renders
app.py - Flask routes → About page: renders
app.py - Flask routes → Uploaded images: saves image
README.md - Flask routes → Inception-BiLSTM: dispatches to
README.md - Flask routes → VGG19-LSTM: dispatches to
README.md - Flask routes → BLIP captioning: dispatches to
README.md - Inception-BiLSTM → Inception features: extracts features
app.py - Inception-BiLSTM → BiLSTM decoding: generates caption
README.md - VGG19-LSTM → VGG19 features: extracts features
app.py - VGG19-LSTM → LSTM decoding: generates caption
README.md - BLIP captioning → Hugging Face: loads pretrained weights
app.py - Flickr30k dataset → BiLSTM experiment: trains on
README.md - Flickr30k dataset → VGG-LSTM experiment: trains on
README.md - Flickr30k dataset → Transformer experiment: evaluates on
README.md - BiLSTM experiment → BLEU evaluation: compares captions
README.md - VGG-LSTM experiment → BLEU evaluation: compares captions
README.md - Transformer experiment → BLEU evaluation: compares captions
README.md - BLIP captioning → BLEU evaluation: reported evaluation
README.md