How vladman-25/image-captioning-experiments works
Main Components
A. Data Pipeline
• Input Layer (Images from Flickr30k dataset)
• Text Processing Layer (Romanian language captions)
• Word Embedding Components:
• Custom Embeddings (FastText, Word2Vec, RoBERT)
• Multiple embedding variations (CC, CoRoLa, NLPL)
B. Model Architecture
• Feature Extraction:
• EfficientNetB4 (as indicated by weights folder)
• Caption Generation Models:
• RNN-based architecture
• Transformer-based architecture
• Multiple Model Variants (as shown by different notebooks)
C. Inference Pipeline
• Dedicated inference component (shown by Inference notebook)
[Input Layer]
• ↓
[Feature Extraction: EfficientNetB4]
• ↓
[Embedding Layer]
• ↙ ↓ ↘
[FastText] [Word2Vec] [RoBERT]
• ↘ ↓ ↙
[Caption Generation Models]
• ↙ ↘
• [RNN] [Transformer]
• ↘ ↙
[Inference Engine]
• ↓
[Output: Romanian Captions]
Important Details to Highlight
A. Multiple Embedding Paths:
• Parallel paths for different embedding types
• Which embedding is used with which architecture
B. Model Variants:
• Clearly separate RNN and Transformer approaches
• How they connect to the embedding layers
C. Training vs. Inference:
Additional Elements
• Different line types (training vs. inference)
• Embedding types
B. Add annotations for:
• Input image specifications
• Output caption format
• Model checkpoint locations
Technical Details to Note
• EfficientNetB4 weights location
• Different notebook implementations
• Dataset connection to Flickr30k-Romanian
Implementation Notes
• Use standardized UML or similar notation
• Keep components proportional to their importance
• Version information where relevant
• Clear data flow directions