# gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures architecture

> Architecture diagram of the GitHub repository gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-02). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures
- Repository: https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures

## Overview

##### Project type and overall purpose
- This is a machine-learning driven application that incorporates Retrieval-Augmented Generation (RAG) techniques to generate Vietnamese poetry. It includes data preprocessing, model training (using LSTM and Transformer models), and a Flask-based web interface to deliver the results.
- The core functionality revolves around combining ML-based poem generation with a retrieval module (using Faiss) to incorporate existing Vietnamese poems for context and structure.

##### File structure to identify major components
- Backend / Model and RAG Infrastructure:
- Files like app.py, pythonfile.py, solver.py, and config.pt indicate the backend logic.
- The pytorch_model.pt file suggests a pre-trained or intermediate machine learning model.
- The poem-generative.ipynb notebook is used for experimentation or demonstrating the generation process.
- The requirements.txt and Dockerfile show that the system is containerized and built using a set of Python and ML dependencies.
- Static & Templates – Web Interface:
- The static directory (with CSS, JS, fonts, images, and SASS files) alongside the templates folder (with at least an index.html) are clearly related to the Flask web interface.
- Data & Dependencies:
- The vocab.pkl, configuration files (.env, config.pt), and the dataset (implicitly referenced through README and used in the RAG system) show the dependency on external data for building and training the ML models.
- Deployment and Environment:
- The Dockerfile and .env indicate that the application is set up with environment-specific configurations and containerization for ease of deployment.

##### Key architectural components
a. Data Sources and Preprocessing:
- A component that represents the Vietnamese poems dataset (mentioned in the README) which feeds into the data processing and model training pipelines.
b. Model Training & Inference:
- An ML module that covers the training phases for LSTM and Transformer models.
- Use of PyTorch and scikit-learn libraries and the role of the pytorch_model.pt.
c. RAG Component:
- A retrieval system that uses Faiss for indexing and querying the poem dataset. This component should interact with the generation model to provide context and structure enhancement.
d. API Integrations:
- External API modules or calls to services from OpenAI, NVIDIA, and Meta, which are integrated to augment the generation process. Show their connection to the core ML and RAG components.
e. Web Interface (Flask Application):
- The Flask app (in app.py) that serves the front-end content using templates, static assets, and routes for user interaction.
f. Deployment & Containerization:

- Data flow from the dataset (poems & vocab) to preprocessing and then feeding into the ML model training pipeline.
- External API calls (OpenAI, NVIDIA, Meta) to the generation module as augmentation inputs.
- Flask application component as the user-facing layer that collects requests, triggers the ML/RAG process, and then returns a generated poem.

##### Architectural Patterns and Design Principles
- Modularity: Separate components for data processing, ML model training/inference, RAG retrieval, and the web interface.
- Separation of concerns, ensuring that the frontend (static assets and templates) is separate from the ML/RAG backend.
- Use clear labels and directional flows to indicate how data moves from one component to the next.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% Data & Dependencies Subgraph
    subgraph "Data & Dependencies"
        dataset["Vietnamese Poems Dataset"]:::data
        vocab["vocab.pkl"]:::data
    end

    %% Backend / ML & RAG Subgraph
    subgraph "Backend / ML & RAG"
        app_py["app.py"]:::ml
        pythonfile_py["pythonfile.py"]:::ml
        solver_py["solver.py"]:::ml
        config_pt["config.pt"]:::ml
        pytorch_model_pt["pytorch_model.pt"]:::ml
        poem_notebook["poem-generative.ipynb"]:::ml
        requirements_txt["requirements.txt"]:::ml
    end

    %% Web Interface Subgraph
    subgraph "Web Interface"
        templates["templates (index.html)"]:::web
        static["static (css,js,fonts,images,sass)"]:::web
    end

    %% Deployment Subgraph
    subgraph "Deployment"
        docker_file["Dockerfile"]:::deploy
        deploy_env[".env"]:::deploy
    end

    %% External APIs Node
    external_APIs["External APIs (OpenAI,NVIDIA,Meta)"]:::external

    %% Data flows to Backend
    dataset -->|"feeds"| app_py
    vocab -->|"provides context"| app_py
    dataset -->|"preprocessed by"| pythonfile_py

    %% Internal Backend interactions
    app_py -->|"orchestrates"| solver_py
    pythonfile_py -->|"supports"| solver_py
    config_pt -->|"configures"| pytorch_model_pt
    solver_py -->|"utilizes"| pytorch_model_pt

    %% External API Integration into Backend (RAG Enhancement)
    external_APIs -->|"augments"| solver_py

    %% Backend output to Web Interface
    solver_py -->|"renders output to"| templates
    solver_py -->|"serves static content to"| static
    templates -->|"sends user requests to"| app_py

    %% Deployment encapsulation
    docker_file --- app_py
    docker_file --- pythonfile_py
    docker_file --- solver_py
    docker_file --- templates
    docker_file --- static
    deploy_env --- docker_file

    %% Web Interface internal connection
    templates -->|"combines with"| static

    %% Click Events for Backend / Model and RAG Infrastructure
    click app_py "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/blob/main/Finale/app.py"
    click pythonfile_py "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/blob/main/Finale/pythonfile.py"
    click solver_py "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/blob/main/Finale/solver.py"
    click config_pt "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/blob/main/Finale/config.pt"
    click pytorch_model_pt "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/blob/main/Finale/pytorch_model.pt"
    click poem_notebook "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/blob/main/Finale/poem-generative.ipynb"
    click requirements_txt "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/blob/main/Finale/requirements.txt"

    %% Click Events for Data & Dependencies
    click vocab "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/blob/main/Finale/vocab.pkl"
    click deploy_env "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/blob/main/Finale/.env"

    %% Click Events for Web Interface
    click templates "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/tree/main/Finale/templates"
    click static "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/tree/main/Finale/static"

    %% Click Event for Deployment
    click docker_file "https://github.com/gpl-ebih/applying-rag-to-enhance-vietnamese-poetry-generation-with-correct-structures/tree/main/Finale/Dockerfile"

    %% Styles
    classDef ml fill:#a2d2ff,stroke:#333,stroke-width:2px;
    classDef web fill:#ffc8dd,stroke:#333,stroke-width:2px;
    classDef data fill:#d0f4de,stroke:#333,stroke-width:2px;
    classDef deploy fill:#ffe066,stroke:#333,stroke-width:2px;
    classDef external fill:#ffafcc,stroke:#333,stroke-width:2px;
```
