# niharikachaudhari13/image-captioning-generator architecture

> Architecture diagram of the GitHub repository niharikachaudhari13/image-captioning-generator, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-07-09). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/niharikachaudhari13/image-captioning-generator
- Repository: https://github.com/niharikachaudhari13/image-captioning-generator

## Overview

##### Project type and purpose
- This is a machine‐learning application with both a Web UI and a command‐line interface.
- Purpose: generate natural‐language captions for user‐provided images using a CNN+LSTM model, support multiple decoding strategies, translation, and text‐to‐speech.

##### File structure
- Top level:
- main.py     → training pipeline (load data, train Xception+LSTM, save model)
- app.py      → Streamlit web front end (user uploads, caption display, download options, TTS)
- test.py     → CLI test harness (batch or single‐image captioning using different strategies)
- requirements.txt → dependency list (Streamlit, TensorFlow/Keras, gTTS, numpy, pandas…)
- README.md   → high‐level project description and usage
- No nested directories; conceptual modules live inside these scripts.
- External dependencies: pretrained Xception weights, Flickr8k dataset, gTTS service.

##### README for additional insights
- Architecture: encoder (Xception without top) extracts 2048D features → decoder (Embedding + LSTM + Dense).
- Captioning options: greedy search, beam search, nucleus sampling.
- Translation to four languages.
- TTS via gTTS.
- UI based on Streamlit.
- Workflow: download data → run main.py to train → run test.py or app.py.

a. Main components
- Data Store: Flickr8k images + captions
- Training Pipeline (main.py)
- Trained Model Artifacts (encoder weights + decoder weights + tokenizer)
- Feature Extraction Service (Xception)
- Caption Generation Service (LSTM with decoding strategies)
- Translation Module
- Text‐to‐Speech Module (gTTS API)
- Streamlit UI (app.py)
- CLI Test Module (test.py)
b. Relationships and interactions
1. Training: Data Store → Training Pipeline → Model Artifacts
2. Inference (UI): User uploads image → Feature Extraction → Caption Generation → (optional) Translation → (optional) TTS → UI display/download
3. Inference (CLI): test.py loads Model Artifacts → runs same Feature Extraction & Caption Generation → outputs .txt/.csv
c. Architectural patterns/design principles
- Separation of concerns: Training vs. Inference vs. Presentation
- Modular pipeline: distinct services for extraction, decoding, translation, TTS
- Reuse of pretrained CNN as encoder
d. Key technologies/frameworks
- TensorFlow/Keras (Xception, LSTM)
- Streamlit (UI)
- gTTS (TTS)
- Python scripting for CLI and training

##### Details to capture
- Input formats (raw image → tensor → 2048D feature vector).
- Decoding strategies as parallel paths or a decision node.
- Model artifacts storage location.
- Deployment context: local execution for both UI and CLI.

## Diagram (Mermaid)

```mermaid
graph TD
    %% Data Sources
    DS["Flickr8k Dataset"]:::data
    PW["Pretrained Weights (Xception)"]:::external

    %% Training Pipeline
    subgraph "Training Pipeline"
        TP["Training Pipeline\n(main.py)"]:::compute
        MA["Model Artifacts\n(encoder & decoder weights, tokenizer)"]:::data
    end

    %% Inference Pipeline
    subgraph "Inference Pipeline"
        EX["Feature Extraction Service\n(Xception encoder)"]:::compute
        CG["Caption Generation Service\n(LSTM decoder: greedy, beam, nucleus)"]:::compute
        TR["Translation Module\n(multi-language)"]:::compute
        TTS["Text-to-Speech Module\n(gTTS)"]:::compute
    end

    %% Presentation Layer
    subgraph "Presentation Layer"
        UI["Streamlit UI\n(app.py)"]:::presentation
        CLI["CLI Test Module\n(test.py)"]:::presentation
    end

    %% External Services & Docs
    GTTS["gTTS API"]:::external
    REQ["requirements.txt"]:::docs
    RD["README.md"]:::docs

    %% Data Flow
    DS --> TP
    PW --> TP
    TP --> MA
    MA --> EX
    EX --> CG
    CG --> TR
    CG --> TTS
    CG --> UI
    CG --> CLI
    MA --> CLI
    TR --> UI
    TTS --> UI
    GTTS --> TTS

    %% Click Events
    click TP "https://github.com/niharikachaudhari13/image-captioning-generator/blob/main/main.py"
    click UI "https://github.com/niharikachaudhari13/image-captioning-generator/blob/main/app.py"
    click CLI "https://github.com/niharikachaudhari13/image-captioning-generator/blob/main/test.py"
    click REQ "https://github.com/niharikachaudhari13/image-captioning-generator/blob/main/requirements.txt"
    click RD "https://github.com/niharikachaudhari13/image-captioning-generator/blob/main/README.md"

    %% Styles
    classDef data fill:#D0E6A5,stroke:#333,stroke-width:1px
    classDef compute fill:#A8E6CF,stroke:#333,stroke-width:1px
    classDef presentation fill:#FFD3B5,stroke:#333,stroke-width:1px
    classDef external fill:#D291BC,stroke:#333,stroke-width:1px
    classDef docs fill:#F5E6CA,stroke:#333,stroke-width:1px
```
