# thakursaabhay/jobsearchrag architecture

> Architecture diagram of the GitHub repository thakursaabhay/jobsearchrag, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-22). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/thakursaabhay/jobsearchrag
- Repository: https://github.com/thakursaabhay/jobsearchrag

## Overview

##### Project Type and Purpose
- Jōbot is a single‐page web application (monolithic) built on Streamlit.
- It provides semantic job searches (via FAISS + embeddings), AI‐driven insights (via OpenAI GPT), and interactive visualizations (via Plotly).

##### Break Down the File Structure
- app.py: the main Streamlit application (UI + orchestration).
- faiss_index.bin & job_metadata.pkl: prebuilt data resources (vector index + job metadata).
- hirist.csv: raw job data used to build the index (used offline or at setup).
- Notebooks (.ipynb): exploration and prototyping, not part of runtime.

##### Derive Key Architectural Components
a. Client / UI Layer
- Streamlit front end (runs in the browser).
- Inputs: API key, search query; Outputs: job results + charts.
b. Application Logic Layer
- Query handling: loads user input.
- Embedding generation: SentenceTransformer model produces vector.
- FAISS similarity search: finds top‐k matching job vectors from faiss_index.bin.
- Metadata lookup: job_metadata.pkl maps vector IDs to job details.
- AI Insights: calls OpenAI’s API for trend analysis or deeper Q&A.
- Data processing & visualization: Pandas aggregates data; Plotly renders charts.
c. Data Storage Layer
- Local files: faiss_index.bin, job_metadata.pkl, hirist.csv.
d. External Service
- OpenAI API (cloud service) for advanced NLP and analytics.

##### Interactions and Data Flow
1. User → (browser) → Streamlit UI
2. Streamlit UI → (function call) → Embedding Service (SentenceTransformer)
3. Embedding → FAISS Engine → returns nearest‐neighbor IDs
4. IDs → Metadata Store (job_metadata.pkl) → returns job records
5. Records → Pandas → interim data structures
6. Optionally: interim data → OpenAI API → returns AI‐driven insights
7. Data + Insights → Plotly → Visualization elements
8. Streamlit UI ← displays results & charts to User

##### Architectural Patterns & Design Principles
- Layered architecture: UI layer decoupled from logic and data layers.
- Reuse of prebuilt data stores (index + metadata) for performance.
- Externalization of heavy‐compute NLP to OpenAI and local FAISS for nearest‐neighbor lookup.
- Single monolithic deployment via Streamlit—no microservices.

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% User Layer
    User["User"]:::user
    Streamlit["Streamlit UI & Orchestration"]:::ui
    User -->|"inputs: API key, query"| Streamlit

    %% Application Logic Layer
    Sentence["Embedding Service (SentenceTransformer)"]:::logic
    FAISS["FAISS Similarity Search Engine"]:::storage
    Metadata["Job Metadata Store"]:::storage
    Pandas["Data Processing (Pandas)"]:::logic
    OpenAI["OpenAI API"]:::external
    Plotly["Visualization (Plotly)"]:::logic

    Streamlit -->|"generate embedding"| Sentence
    Sentence -->|"vector"| FAISS
    FAISS -->|"ids"| Metadata
    Metadata -->|"records"| Pandas
    Pandas -->|"request insights"| OpenAI
    OpenAI -->|"insights"| Pandas
    Pandas -->|"prepare data"| Plotly
    Plotly -->|"charts"| Streamlit

    %% Data Storage Layer
    subgraph "Local Data Stores"
        FAISS
        Metadata
    end

    %% Offline Build Pipeline
    subgraph "Offline Index-Build Pipeline"
        CSV["Raw Job Data (hirist.csv)"]:::storage
        EmbedBuild["Embedding Generation"]:::logic
        BuildIndex["Build FAISS Index & Metadata"]:::storage
        CSV --> EmbedBuild
        EmbedBuild --> BuildIndex
        %% Notebooks
        Notebook1["HiristScraping.ipynb"]:::notebook
        Notebook2["JobSearchAgent.ipynb"]:::notebook
        Notebook3["RAGJobSearch.ipynb"]:::notebook
        Notebook1 --> CSV
        Notebook2 --> CSV
        Notebook3 --> CSV
    end

    %% Documentation
    Doc["Documentation & Setup (README.md)"]:::doc

    %% Click Events
    click Streamlit "https://github.com/thakursaabhay/jobsearchrag/blob/main/app.py"
    click FAISS "https://github.com/thakursaabhay/jobsearchrag/blob/main/faiss_index.bin"
    click Metadata "https://github.com/thakursaabhay/jobsearchrag/blob/main/job_metadata.pkl"
    click CSV "https://github.com/thakursaabhay/jobsearchrag/blob/main/hirist.csv"
    click Notebook1 "https://github.com/thakursaabhay/jobsearchrag/blob/main/HiristScraping.ipynb"
    click Notebook2 "https://github.com/thakursaabhay/jobsearchrag/blob/main/JobSearchAgent.ipynb"
    click Notebook3 "https://github.com/thakursaabhay/jobsearchrag/blob/main/RAGJobSearch.ipynb"
    click Doc "https://github.com/thakursaabhay/jobsearchrag/blob/main/README.md"

    %% Styles
    classDef user fill:#E0E0E0,stroke:#999,stroke-width:1px
    classDef ui fill:#AED6F1,stroke:#2E86C1,stroke-width:2px
    classDef logic fill:#A9DFBF,stroke:#27AE60,stroke-width:2px
    classDef storage fill:#F8C471,stroke:#D35400,stroke-width:2px,shape:cylinder
    classDef external fill:#D7BDE2,stroke:#8E44AD,stroke-width:2px
    classDef notebook fill:#F0F0F0,stroke:#AAA,stroke-dasharray: 5 5
    classDef doc fill:#FCF3CF,stroke:#B7950B,stroke-width:2px
```
