# hungreeee/resume-screening-rag-pipeline architecture

> Architecture diagram of the GitHub repository hungreeee/resume-screening-rag-pipeline, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-01-20). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/hungreeee/resume-screening-rag-pipeline
- Repository: https://github.com/hungreeee/resume-screening-rag-pipeline

## Overview

##### Component Breakdown

Frontend Layer:
- It to API/Service layer below

Processing Layer (Core):
- Create three main processing blocks:
1. Query Processing Block:
- Query Classifier
- Function Calling Module
- Chat History Manager

2. RAG Pipeline Block (Central):
- Sub-query Generator
- Retriever Module (with two sub-components):
- RAG Retriever
- RAG Fusion Retriever
- Document Re-ranker
- Context Augmentation Module

3. LLM Generation Block:
- LLM Agent (OpenAI)
- Response Generator

Data Storage Layer:
- Vector Stores (FAISS):
- PDF Vector Store
- Synthetic Vector Store
- Document Storage:
- Resume Database
- Job Descriptions
- Evaluation Results

##### Integration Points

Clear connections between:
- Vector stores and retrieval components
- LLM services and generation components
- Storage systems and processing modules

##### Key Technical Elements to Highlight

- Langchain
- OpenAI
- HuggingFace
- FAISS
- Streamlit

##### Additional Elements

Include:
- Version/environment indicators
- External service connections
- System boundaries

##### Performance Flow

"Small-to-Big retrieval" process:
- Text chunk processing
- Document retrieval
- Full-text augmentation

## Diagram (Mermaid)

```mermaid
graph TB
    %% Styles and classes
    classDef frontend fill:#3498db,stroke:#2980b9,color:white
    classDef processing fill:#e67e22,stroke:#d35400,color:white
    classDef storage fill:#2ecc71,stroke:#27ae60,color:white
    classDef api fill:#9b59b6,stroke:#8e44ad,color:white
    classDef llm fill:#e74c3c,stroke:#c0392b,color:white

    %% Frontend Layer
    subgraph Frontend
        UI["User Interface (Streamlit)"]
    end
    click UI "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/blob/main/demo/interface.py"

    %% Processing Layer
    subgraph Processing
        %% Query Processing Block
        subgraph QueryBlock["Query Processing"]
            QC["Query Classifier"]
            FCM["Function Calling Module"]
            CHM["Chat History Manager"]
        end

        %% RAG Pipeline Block
        subgraph RAGBlock["RAG Pipeline"]
            SQG["Sub-query Generator"]
            
            subgraph RetrieverModule["Retriever Module"]
                RR["RAG Retriever"]
                RF["RAG Fusion Retriever"]
            end
            click RetrieverModule "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/blob/main/demo/retriever.py"
            
            DR["Document Re-ranker"]
            CAM["Context Augmentation"]
        end

        %% LLM Generation Block
        subgraph LLMBlock["LLM Generation"]
            LLMA["LLM Agent (OpenAI)"]
            RG["Response Generator"]
        end
        click LLMA "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/blob/main/demo/llm_agent.py"
    end

    %% Data Storage Layer
    subgraph Storage
        subgraph VectorStores["Vector Stores"]
            PVS["PDF Vector Store"]
            SVS["Synthetic Vector Store"]
        end
        click PVS "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/tree/main/vectorstore-pdf/"
        click SVS "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/tree/main/vectorstore-synthetic/"

        subgraph DocStorage["Document Storage"]
            RDB["Resume Database"]
            JD["Job Descriptions"]
            ER["Evaluation Results"]
        end
        click RDB "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/tree/main/data/supplementary-data/pdf-resumes/"
        click JD "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/tree/main/data/main-data/data-checkpoints/"
        click ER "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/tree/main/data/main-data/gpt35-normalrag/evaluation-results/"

        subgraph TestResults["Test Results"]
            RRT["RAG Retriever Tests"]
            RFT["RAG Fusion Tests"]
        end
        click RRT "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/tree/main/data/main-data/rag-retriever/test-results/"
        click RFT "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/tree/main/data/main-data/rag-fusion-retriever/test-results/"
    end

    %% Data Preprocessing
    DI["Data Ingestion"]
    DP["Data Preprocessing"]
    ES["Evaluation System"]
    click DI "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/blob/main/demo/interactive/ingest_data.py"
    click DP "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/blob/main/preprocessing/data_cleaning.ipynb"
    click ES "https://github.com/Hungreeee/Resume-Screening-RAG-Pipeline/blob/main/evaluation/ragas_evaluation.ipynb"

    %% Connections
    UI --> QC
    QC --> FCM
    FCM --> CHM
    CHM --> SQG
    SQG --> RetrieverModule
    RetrieverModule --> DR
    DR --> CAM
    CAM --> LLMA
    LLMA --> RG
    RG --> UI

    VectorStores --> RetrieverModule
    DocStorage --> DI
    DI --> DP
    DP --> VectorStores
    RetrieverModule --> TestResults
    RetrieverModule --> ES
    ES --> ER

    %% Apply styles
    class UI frontend
    class QC,FCM,CHM,SQG,RetrieverModule,DR,CAM processing
    class LLMA,RG llm
    class PVS,SVS,RDB,JD,ER,RRT,RFT storage
    class DI,DP,ES api
```
