How thakursaabhay/jobsearchrag works
Project Type and Purpose
• Jōbot is a single‐page web application (monolithic) built on Streamlit.
• It provides semantic job searches (via FAISS + embeddings), AI‐driven insights (via OpenAI GPT), and interactive visualizations (via Plotly).
Break Down the File Structure
• app.py: the main Streamlit application (UI + orchestration).
• faiss_index.bin & job_metadata.pkl: prebuilt data resources (vector index + job metadata).
• hirist.csv: raw job data used to build the index (used offline or at setup).
• Notebooks (.ipynb): exploration and prototyping, not part of runtime.
Derive Key Architectural Components
a. Client / UI Layer
• Streamlit front end (runs in the browser).
• Inputs: API key, search query; Outputs: job results + charts.
b. Application Logic Layer
• Query handling: loads user input.
• Embedding generation: SentenceTransformer model produces vector.
• FAISS similarity search: finds top‐k matching job vectors from faiss_index.bin.
• Metadata lookup: job_metadata.pkl maps vector IDs to job details.
• AI Insights: calls OpenAI’s API for trend analysis or deeper Q&A.
• Data processing & visualization: Pandas aggregates data; Plotly renders charts.
c. Data Storage Layer
• Local files: faiss_index.bin, job_metadata.pkl, hirist.csv.
d. External Service
• OpenAI API (cloud service) for advanced NLP and analytics.
Interactions and Data Flow
1. User → (browser) → Streamlit UI
2. Streamlit UI → (function call) → Embedding Service (SentenceTransformer)
3. Embedding → FAISS Engine → returns nearest‐neighbor IDs
4. IDs → Metadata Store (job_metadata.pkl) → returns job records
5. Records → Pandas → interim data structures
6. Optionally: interim data → OpenAI API → returns AI‐driven insights
7. Data + Insights → Plotly → Visualization elements
8. Streamlit UI ← displays results & charts to User
Architectural Patterns & Design Principles
• Layered architecture: UI layer decoupled from logic and data layers.
• Reuse of prebuilt data stores (index + metadata) for performance.
• Externalization of heavy‐compute NLP to OpenAI and local FAISS for nearest‐neighbor lookup.
• Single monolithic deployment via Streamlit—no microservices.