# sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag- architecture

> Architecture diagram of the GitHub repository sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-
- Repository: https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-

## Overview

##### Project type and purpose
- This is a full-stack RAG (Retrieval-Augmented Generation) application.
- Purpose: Provide a chat interface (React) that interacts with a Node.js REST API, which in turn calls a Python Flask service for document retrieval (via Milvus) and LLM calls (Google’s Gem­ini API + optional SERP API).

##### File structure
- Top level:
- public/, src/ → React front-end
- server/ → Node.js REST API
- Python-Server/ → Flask-based retrieval & LLM microservice
- SyntheticDatasetforIntentClassifier/ → offline data‐generation scripts
- server/: MVC-style folders (models/, controllers/, routes/, middleware/), config/db.js for MongoDB, utils for email/encryption.
- Python-Server/: Flask entrypoint (app.py), intent-classification pipelines, retrieval modules, gemini_api.py, decryption utilities, embedding ingestion.
- Config files: package.json/npm lock for Node projects, requirements implied by README, .env for environment variables.

##### README for additional insights
- Tech stack: React, Node.js+Express, Python+Flask, MongoDB, Milvus, Google Generative AI, SERP API.
- Ports & interaction flow: React (3000) → Node API (4000) → Flask service (5000).
- External dependencies: MongoDB, Milvus vector database, email SMTP, Google API.

## Diagram (Mermaid)

```mermaid
graph TB
    %% UI Layer
    subgraph "UI Layer"
        ReactApp["React App (3000)"]:::frontend
    end

    %% API Layer
    subgraph "API Layer"
        ExpressAPI["Express API (4000)"]:::backend
        subgraph "Controllers"
            AuthCtrl["authController.js"]:::backend
            ConvCtrl["conversationController.js"]:::backend
            UserCtrl["userController.js"]:::backend
        end
        subgraph "Routes"
            AuthRoutes["authRoutes.js"]:::backend
            ConvRoutes["conversationRoutes.js"]:::backend
            UserRoutes["userRoutes.js"]:::backend
        end
        subgraph "Models"
            UserModel["User.js"]:::backend
            ConvModel["Conversation.js"]:::backend
        end
        subgraph "Middleware"
            AuthMid["authMiddleware.js"]:::backend
            UploadMid["uploadMiddleware.js"]:::backend
        end
        subgraph "Utils"
            EmailUtil["emailService.js"]:::backend
            EncryptUtil["encryption.js"]:::backend
        end
        DBConfig["db.js"]:::backend
    end

    %% ML Service Layer
    subgraph "Flask RAG Service (5000)"
        FlaskApp["app.py"]:::backend
        subgraph "Retrieval"
            Retriever["retriever.py"]:::backend
            Ingest["ingest_embeddings.py"]:::backend
        end
        subgraph "LLM & Intent"
            GeminiClient["gemini_api.py"]:::backend
            SERPClient["serpAPI.py"]:::backend
            DecryptUtil["decryption_util.py"]:::backend
            EvalIntent["EvaluationIntent.py"]:::backend
            FineTune1["FineTunedIntentClassifierModel.py"]:::backend
            FineTune2["IntentClassifierFinetuningLoRA.py"]:::backend
            EvalScript["Intent_Classifier_eval.py"]:::backend
        end
    end

    %% Offline Data-gen
    subgraph "Offline Data-gen"
        DataGen["SyntheticDataSetGenerator.py"]:::backend
        CSVMod["csv_modifier.py"]:::backend
        QExtract["question_extractor.py"]:::backend
    end

    %% Datastores
    MongoDB["MongoDB"]:::db
    Milvus["Milvus"]:::db

    %% External Services
    GeminiAPI["Gemini API"]:::external
    SERPAPI["SERP API"]:::external
    SMTP["SMTP Email"]:::external

    %% Connections
    ReactApp -->|"HTTP"| ExpressAPI
    ExpressAPI -->|"Internal HTTP"| FlaskApp
    FlaskApp -->|"gRPC/TCP"| Milvus
    FlaskApp -->|"HTTPS"| GeminiAPI
    FlaskApp -.->|"HTTP optional"| SERPAPI
    ExpressAPI -->|"MongoDB Driver"| MongoDB
    ExpressAPI -->|"SMTP"| SMTP

    %% Internal API structure flows
    ExpressAPI -->|"/api/auth"| AuthRoutes
    ExpressAPI -->|"/api/conversations"| ConvRoutes
    ExpressAPI -->|"/api/users"| UserRoutes

    AuthRoutes --> AuthCtrl
    ConvRoutes --> ConvCtrl
    UserRoutes --> UserCtrl

    AuthCtrl --> UserModel
    ConvCtrl --> ConvModel
    UserCtrl --> UserModel

    AuthCtrl --> EncryptUtil
    AuthCtrl --> EmailUtil

    ExpressAPI --> DBConfig

    FlaskApp --> Retriever
    FlaskApp --> Ingest
    FlaskApp --> GeminiClient
    FlaskApp --> SERPClient
    FlaskApp --> DecryptUtil

    %% Click Events
    click ReactApp "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/tree/main/src/"
    click ExpressAPI "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/server.js"
    click DBConfig "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/config/db.js"
    click UserModel "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/models/User.js"
    click ConvModel "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/models/Conversation.js"
    click AuthCtrl "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/controllers/authController.js"
    click ConvCtrl "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/controllers/conversationController.js"
    click UserCtrl "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/controllers/userController.js"
    click AuthRoutes "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/routes/authRoutes.js"
    click ConvRoutes "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/routes/conversationRoutes.js"
    click UserRoutes "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/routes/userRoutes.js"
    click AuthMid "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/middleware/authMiddleware.js"
    click UploadMid "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/middleware/uploadMiddleware.js"
    click EmailUtil "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/utils/emailService.js"
    click EncryptUtil "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/server/utils/encryption.js"
    click FlaskApp "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/app.py"
    click Retriever "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/retriever.py"
    click Ingest "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/ingest_embeddings.py"
    click GeminiClient "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/gemini_api.py"
    click SERPClient "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/serpAPI.py"
    click DecryptUtil "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/decryption_util.py"
    click EvalIntent "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/EvaluationIntent.py"
    click FineTune1 "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/FineTunedIntentClassifierModel.py"
    click FineTune2 "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/IntentClassifierFinetuningLoRA.py"
    click EvalScript "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/Python-Server/Intent_Classifier_eval.py"
    click DataGen "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/SyntheticDatasetforIntentClassifier/SyntheticDataSetGenerator.py"
    click CSVMod "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/SyntheticDatasetforIntentClassifier/csv_modifier.py"
    click QExtract "https://github.com/sai0499/diffusion-page-reranking-aided-dense-retrieval-for-rag-/blob/main/SyntheticDatasetforIntentClassifier/question_extractor.py"

    %% Styles
    classDef frontend fill:#B3CDE0,stroke:#34495e,color:#000
    classDef backend fill:#6497B1,stroke:#34495e,color:#fff
    classDef db fill:#C4DF9B,stroke:#34495e,color:#000
    classDef external fill:#CCCCCC,stroke:#34495e,color:#000
```
