How abdelkadirsellahi/vqa-using-yolo-bert-gpt-2 works
Component Details
For each major component, create sub-components showing the internal processing:
a) YOLOv8 Block:
• Three output branches:
• Confidence Scores
• Class Values
• A "Feature Extraction" sub-block
b) BERT Block:
• Processing steps:
• Tokenization
• Input IDs
• Attention Masks
• Text Embeddings
c) Feature Fusion Block:
• Combination of:
• Visual Features
• Text Embeddings
• Linear Projection Layer
d) GPT-2 Block:
• Processing of:
• Combined Features Input
• Logits Generation
• Answer Selection
• Blue for image-related data
• Green for text-related data
• Purple for combined features
Additional Elements
a) Processing Details:
• Image size (224x224)
• Batch size (64)
• Hidden dimension (256)
b) Model Parameters:
• Learning rate (1e-5)
• Number of epochs (30)
Technical Details
Annotations for:
• Input/Output formats
• Key transformations
• Model architectures
• Performance metrics
Optimization
• Maintaining proper spacing
• Grouping related components
• Using clear, readable labels
• Training (VQA_YOLO_BERT_GPT2_Training.ipynb)
• Validation (VQA_YOLO_BERT_GPT2_Validation.ipynb)
• Prediction (VQA_YOLO_BERT_GPT2_Prediction.ipynb)