# abbas99-hub/job-recommendation-system architecture

> Architecture diagram of the GitHub repository abbas99-hub/job-recommendation-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-10). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/abbas99-hub/job-recommendation-system
- Repository: https://github.com/abbas99-hub/job-recommendation-system

## Overview

##### Project Type and Purpose
- This is a full-stack, machine-learning-driven job recommendation system.
- Purpose: scrape job data from Glassdoor, clean & engineer features, train a TF-IDF + NearestNeighbors model, expose an API and a Streamlit frontend, and deploy on Azure (Docker, ACR, AKS).

##### Main Architectural Layers & Components
A. Data Ingestion Layer
- Glassdoor Scraper (uses chromedriver)
- Raw Data Store (CSV / JSON in src/data)
B. Data Processing Layer
- jd_data_cleaner.py: missing-value handling, normalization
- jd_data_extractor.py: parse/unstructure job descriptions
- Skill Extractor Utility (Spacy-based PDF skill parsing in utilities)
C. Feature Engineering + Model Training Layer
- job_recommender.py: TF-IDF vectorizer, NearestNeighbors trainer
- Model Artifacts (serialized .pkl or similar)
- Jupyter Notebooks (src/notebook) for EDA & prototyping
D. Serving/API Layer
- Containerized service (e.g. Flask/FastAPI wrapper around model)
- API Endpoints:
- /recommend (accepts user resume or preferences)
- /skills (extract skills from uploaded resume)
E. Presentation Layer
- Streamlit App (__init__.py entrypoint)
- UI Components: file upload, results table, filters
F. Deployment & Infrastructure Layer
- Dockerfile (package model + API)
- Azure Container Registry (store images)
- Azure Kubernetes Service (or Azure App Service)
- Monitoring & Logging (Azure Monitor / Application Insights)

##### Relationships & Data Flows
1. Glassdoor → Scraper → Raw CSV/JSON
2. Raw Data → Cleaner/Extractor → Structured Feature Set
3. Structured Data → TF-IDF + NearestNeighbors → Model Artifact
4. Model Artifact + API Code → Docker Container → ACR → AKS
5. Streamlit UI ↔ API Service
6. User uploads resume → Skill Extractor → skills vector → /recommend → recommendations

##### Architectural Patterns & Design Principles
- Separation of Concerns: clearly separate ingestion, processing, modeling, API, and UI.
- Microservice-style Deployment: model and API in a container, scalable via AKS.
- CI/CD Ready: container registry + Kubernetes for automated builds and rollouts.

##### Technology Stack
- Python 3.10
- Scraping: Selenium + ChromeDriver
- Data Processing: pandas, numpy
- NLP: scikit-learn (TF-IDF, NearestNeighbors), SpaCy
- Frontend: Streamlit
- Packaging: setup.py, requirements.txt, Docker
- Cloud: Azure Container Registry, Azure Kubernetes Service
- Notebooks: Jupyter

- All six layers (ingestion, processing, modeling, API, UI, deployment)
- External dependencies (Glassdoor, Azure)
- Data stores (CSV/JSON, serialized model)
- Execution flow: from scraping to user recommendation
- Real-time path: resume upload → skill extraction → recommendation API → UI

## Diagram (Mermaid)

```mermaid
flowchart TD
    %% External Sources
    Glassdoor["Glassdoor"]:::external
    User["User"]:::external

    %% Data Ingestion Layer
    subgraph "Data Ingestion Layer"
        direction TB
        GDScraper["Glassdoor Scraper\n(chromedriver)"]:::process
        RawStore["Raw Data Store\n(CSV/JSON)"]:::data
    end

    %% Data Processing Layer
    subgraph "Data Processing Layer"
        direction TB
        Cleaner["Job Description Cleaner"]:::process
        Extractor["Job Description Extractor"]:::process
        SkillUtil["Skill Extraction Utility"]:::utility
    end

    %% Feature Engineering & Model Training Layer
    subgraph "Model Training Layer"
        direction TB
        Recommender["Job Recommender\n(TF-IDF + NearestNeighbors)"]:::process
        ModelArtifacts["Model Artifacts\n(.pkl, etc.)"]:::data
        Notebooks["Jupyter Notebooks\n(EDA & Prototyping)"]:::document
    end

    %% Serving/API Layer
    subgraph "API Service Layer"
        direction TB
        APIService["Model API Service\n(Flask/FastAPI)"]:::service
        EPRecommend["POST /recommend"]:::endpoint
        EPSkills["POST /skills"]:::endpoint
    end

    %% Presentation Layer
    subgraph "Presentation Layer"
        direction TB
        StreamlitApp["Streamlit App"]:::service
        ResumeDoc["Sample Resumes\n(CV.pdf)"]:::document
        UIComponents["UI Components\n(file upload, filters)"]:::ui
    end

    %% Deployment & Infrastructure Layer
    subgraph "Deployment & Infrastructure"
        direction TB
        Dockerfile["Dockerfile"]:::config
        ACR["Azure Container Registry"]:::cloud
        AKS["Azure Kubernetes Service"]:::cloud
        Monitor["Monitoring & Logging\n(Azure Monitor)"]:::cloud
        Requirements["requirements.txt"]:::config
        Setup["setup.py"]:::config
    end

    %% Flows
    Glassdoor -->|scrapes| GDScraper -->|stores raw| RawStore
    RawStore -->|input| Cleaner -->|clean data| Extractor -->|structured features| Recommender -->|trained model| ModelArtifacts
    Notebooks -.-> Recommender
    ResumeDoc -->|upload| SkillUtil -->|skills vector| EPSkills
    EPSkills -->|extract skills| APIService
    UIComponents --> StreamlitApp
    StreamlitApp <-->|requests| APIService
    EPRecommend --> APIService
    APIService -->|uses model| ModelArtifacts
    ModelArtifacts & APIService --> Dockerfile --> ACR --> AKS
    AKS --> Monitor

    %% Click Events
    click GDScraper "https://github.com/abbas99-hub/job-recommendation-system/blob/main/./chromedriver_win32/chromedriver.exe"
    click RawStore "https://github.com/abbas99-hub/job-recommendation-system/tree/main/./src/data/"
    click Cleaner "https://github.com/abbas99-hub/job-recommendation-system/blob/main/./src/components/jd_data_cleaner.py"
    click Extractor "https://github.com/abbas99-hub/job-recommendation-system/blob/main/./src/components/jd_data_extractor.py"
    click SkillUtil "https://github.com/abbas99-hub/job-recommendation-system/blob/main/./src/notebook/skills_extraction.py"
    click Recommender "https://github.com/abbas99-hub/job-recommendation-system/blob/main/./src/components/job_recommender.py"
    click Notebooks "https://github.com/abbas99-hub/job-recommendation-system/tree/main/./src/notebook/"
    click ResumeDoc "https://github.com/abbas99-hub/job-recommendation-system/blob/main/./utilities/resumes/CV.pdf"
    click Requirements "https://github.com/abbas99-hub/job-recommendation-system/blob/main/./requirements.txt"
    click Setup "https://github.com/abbas99-hub/job-recommendation-system/blob/main/./setup.py"

    %% Styles
    classDef process fill:#AEDFF7,stroke:#036B99;
    classDef data fill:#FFC966,stroke:#995C00;
    classDef utility fill:#C4E1C1,stroke:#2E7D32;
    classDef document fill:#F7D6E0,stroke:#9C2166;
    classDef service fill:#FEEA9A,stroke:#C49F1C;
    classDef endpoint fill:#FFCCCB,stroke:#A12D2D,stroke-dasharray: 2 2;
    classDef ui fill:#DAD2E0,stroke:#7851A9;
    classDef cloud fill:#E0F7FA,stroke:#007C91,stroke-dasharray: 5 5;
    classDef config fill:#E8EAF6,stroke:#3F51B5;
    classDef external fill:#F0F0F0,stroke:#999999,stroke-dasharray: 3 3;
```
