# duongtruongbinh/vivqa-x architecture

> Architecture diagram of the GitHub repository duongtruongbinh/vivqa-x, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-01). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/duongtruongbinh/vivqa-x
- Repository: https://github.com/duongtruongbinh/vivqa-x

## Overview

##### Project Type and Purpose
- This is a research-oriented, end-to-end pipeline designed to construct a Vietnamese Visual Question Answering dataset with natural language explanations (VQA-NLE). Its purpose is not only to build the dataset but also to benchmark different models (heuristic and baseline LSTM-generative) on it.
- The repository includes multiple “subsystems”: dataset management, data processing via an automated pipeline, model benchmarking (training and evaluation), and utilities/scripts for deployment.

##### Main Components
- Data Layer:
- “data/final” holds the core JSON files (train, validation, test) sourced from COCO with Vietnamese questions, answers, and explanations.
- That there’s an initial download process (using the download_vqax.sh script) to retrieve the dataset.

- Pipeline Module:
- Under “src/pipeline”, the project separates functions into several submodules:
▪ Post-Processing: “post_processing” – this might handle refinement of model outputs or dataset quality.
▪ Selection: “selection” – which further contains evaluators (base, gemma, gpt, llama, etc.) for selecting and ranking generated outputs.
▪ Translation: “translation” – modules that involve data processing for translation (with submodules for different translation engines).

- Models Module:
- Contains two main model directories:
▪ Baseline Model (LSTM-Generative): “src/models/baseline_model”
- Contains configuration (config/config.yaml), training (train.py), evaluation (evaluate.py), and a model definition (vivqax_model.py) along with dataloaders, metrics, and utilities.
▪ Heuristic Model: “src/models/heuristic_model”
- Contains similar structure with configuration, dataloaders, heuristics (heuristic_baseline.py) and a run script (run_heuristic.py).
- Separate branches for these models indicating they both share some standard evaluation metrics (e.g., BLEU, ROUGE, SPICE, etc.) but are triggered by different scripts/entry points.

- Configuration and Scripting:
- “requirements.txt” and configuration YAML files supply runtime parameters including device settings, model hyperparameters, and file paths.
- “scripts/” include shell scripts (pipeline.sh, train.sh, evaluate.sh) to execute the automated workflow steps – from data downloading to training and evaluation.
- How these scripts orchestrate the components, i.e., they serve as entry points that trigger the underlying Python modules in “src”.

##### Relationships and Interactions
- Data flows from the data layer into the pipeline module where translation, selection, and post-processing prepare the dataset.
- The processed data is then fed into model training (baseline model’s train.py) or evaluated directly by the heuristic model’s run script.
- Evaluators in the “selection” submodule provide feedback (possibly through metric calculations) that might be used for refining model results.
- Configuration files provide the necessary parameters for both the pipeline operations and model execution.
- That training outputs (model weights) and evaluation results are stored and later used for performance assessment.

##### Architectural Patterns and Design Principles
- Modularity: Each major function (data processing, model training, evaluation, translation, selection) is encapsulated in its own module or directory.
- Config-driven: YAML configuration files drive operational parameters, allowing for flexible experimentation.
- Scripted Orchestration: Shell scripts in the “scripts” folder serve as orchestration wrappers for end-to-end execution.
- Clear separation between rule-based (heuristic) and learning-based (LSTM-generative) approaches.

##### Elements to Feature
- Components (e.g., “Data Ingestion”, “Translation Module”, “Selection Evaluators”, “Baseline Model Training”, “Heuristic Model Evaluation”, “Post Processing”).
- How the raw dataset (data/final) is processed through the pipeline modules.
- Flow from configuration files to the individual modules.
- Labels and Annotations:
- Evaluation metrics flow (e.g., “Output: BLEU, METEOR, etc.”) emanating from model evaluation blocks.
- Visual Distinctions:
- Optionally, incorporate callouts to indicate the shell scripts that orchestrate these components.

## Diagram (Mermaid)

```mermaid
flowchart TD
    A["Download Script"]:::orchestra
    B["Data Source"]:::dataLayer
    C["Pipeline Module"]:::pipeline
    subgraph "Pipeline Processing"
        T["Translation"]:::pipeline
        S["Selection"]:::pipeline
        P["Post-Processing"]:::pipeline
    end
    D["Processed Data"]
    subgraph "Baseline Model (LSTM-Generative)"
        BL_LABEL["Baseline Model"]:::model
        BL1["Config (config.yaml)"]:::model
        BL2["Train Script (train.py)"]:::model
        BL3["Evaluate Script (evaluate.py)"]:::model
        BL4["Model Definition (vivqax_model.py)"]:::model
    end
    subgraph "Heuristic Model"
        HE_LABEL["Heuristic Model"]:::model
        HE1["Config (config.yaml)"]:::model
        HE2["Heuristic Baseline (heuristic_baseline.py)"]:::model
        HE3["Run Script (run_heuristic.py)"]:::model
    end
    E["Orchestration & Config"]:::orchestra

    A -->|"triggers"| B
    B -->|"feeds data to"| C
    C -->|"executes"| T
    C -->|"executes"| S
    C -->|"executes"| P
    T --> D
    S --> D
    P --> D
    D -->|"inputs to"| BL_LABEL
    D -->|"inputs to"| HE_LABEL
    BL_LABEL --> BL1
    BL1 --> BL2
    BL2 --> BL3
    BL3 --> BL4
    HE_LABEL --> HE1
    HE1 --> HE2
    HE2 --> HE3
    E -->|"orchestrates"| C
    E -->|"configures"| BL_LABEL
    E -->|"configures"| HE_LABEL

    click A "https://github.com/duongtruongbinh/vivqa-x/blob/main/scripts/download_vqax.sh"
    click B "https://github.com/duongtruongbinh/vivqa-x/tree/main/data/final"
    click C "https://github.com/duongtruongbinh/vivqa-x/tree/main/src/pipeline"
    click T "https://github.com/duongtruongbinh/vivqa-x/tree/main/src/pipeline/translation"
    click S "https://github.com/duongtruongbinh/vivqa-x/tree/main/src/pipeline/selection"
    click P "https://github.com/duongtruongbinh/vivqa-x/blob/main/src/pipeline/post_processing/post_processing.py"
    click BL_LABEL "https://github.com/duongtruongbinh/vivqa-x/tree/main/src/models/baseline_model"
    click HE_LABEL "https://github.com/duongtruongbinh/vivqa-x/tree/main/src/models/heuristic_model"
    click E "https://github.com/duongtruongbinh/vivqa-x/tree/main/scripts"

    classDef dataLayer fill:#e0f7fa,stroke:#0077b6,stroke-width:2px;
    classDef pipeline fill:#ffe0b2,stroke:#ff8800,stroke-width:2px;
    classDef model fill:#c8e6c9,stroke:#2e7d32,stroke-width:2px;
    classDef orchestra fill:#d1c4e9,stroke:#6a1b9a,stroke-width:2px;
```
