# manojmeena0001/job_skills_extractor architecture

> Architecture diagram of the GitHub repository manojmeena0001/job_skills_extractor, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-04-19). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/manojmeena0001/job_skills_extractor
- Repository: https://github.com/manojmeena0001/job_skills_extractor

## Overview

##### Project Type and Purpose
- This is a lightweight “full‑stack” Python web application using Streamlit as the front‑end/UI layer and pure Python modules for the back‑end processing pipeline.
- Purpose: Accept a job description via a web interface, run NLP‑based preprocessing, extract skills from text against a predefined list, categorize them, and render results back in the browser.

##### File Structure
- Top‑level
- .streamlit/ – Streamlit configuration for theming, credentials, etc.
- data/ – JSON files: `skill_dictionary.json` (full skill list) and `skill_categories.json` (mapping skills to categories).
- app.py – Streamlit application entry point, UI layout, input/output.
- preprocessing.py – Text cleaning, tokenization, stopword removal.
- skill_extractor.py – Core extraction logic matching tokens to skills.
- skill_categorizer.py – Logic to group extracted skills into categories.
- utils.py – Helper functions (logging, file I/O, common routines).
- pyproject.toml & uv.lock – Dependency management.
- No separate “database”—data is loaded from JSON. No microservices; single monolithic service.

##### Insights from README
- Data Flow Steps:
1. Input via Streamlit UI (app.py)
2. Preprocessing (preprocessing.py)
3. Skill Extraction (skill_extractor.py)
4. Skill Categorization (skill_categorizer.py)
5. Render in UI
- Tech Stack: Python, Streamlit, NLTK, pandas
- Ready for extension into larger ATS/HR platforms, so the pipeline is modular.

- Use clearly labeled rectangles for each module/component.
- UI layer in blue
- Processing pipeline in green
- Data store in orange
- External libraries in gray

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% UI Layer
    subgraph "UI Layer"
        direction TB
        Browser["User’s Browser"]:::ui
        Streamlit["Streamlit Server"]:::ui
        Config["Streamlit Configuration"]:::ui
    end

    %% NLP Pipeline
    subgraph "NLP Pipeline"
        direction TB
        Preprocessing["Preprocessing Module"]:::proc
        SkillExtract["Skill Extraction Module"]:::proc
        SkillCategorizer["Skill Categorization Module"]:::proc
    end

    %% Shared Utilities
    Utils["Shared Utilities"]:::shared

    %% Data Store
    subgraph "Data Store"
        direction TB
        Dict["Skill Dictionary (JSON)"]:::data
        Categories["Skill Categories Mapping (JSON)"]:::data
    end

    %% External Libraries
    subgraph "External Libraries"
        direction TB
        NLTK["NLTK"]:::ext
        Pandas["pandas"]:::ext
    end

    %% Static Assets
    Assets["Attached Static Assets"]:::assets

    %% Connections
    Browser -->|"POST job description"| Streamlit
    Config -->|"theming & credentials"| Streamlit
    Streamlit -->|"raw text"| Preprocessing
    Preprocessing -->|"clean tokens"| SkillExtract
    SkillExtract -->|"extracted skill list"| SkillCategorizer
    SkillCategorizer -->|"categorized skills"| Streamlit
    Streamlit -->|"render results"| Browser

    Preprocessing -->|"read skill lists & categories"| Dict
    Preprocessing -->|"read skill lists & categories"| Categories
    SkillExtract -->|"read skill lists & categories"| Dict
    SkillExtract -->|"read skill lists & categories"| Categories
    SkillCategorizer -->|"read skill lists & categories"| Dict
    SkillCategorizer -->|"read skill lists & categories"| Categories

    Preprocessing -->|"NLTK functions"| NLTK
    SkillExtract -->|"NLTK functions"| NLTK
    SkillCategorizer -->|"pandas DataFrame ops"| Pandas

    Preprocessing --> Utils
    SkillExtract --> Utils
    SkillCategorizer --> Utils
    Streamlit --> Utils

    Streamlit --> Assets

    %% Click Events
    click Streamlit "https://github.com/manojmeena0001/job_skills_extractor/blob/main/app.py"
    click Config "https://github.com/manojmeena0001/job_skills_extractor/blob/main/.streamlit/config.toml"
    click Preprocessing "https://github.com/manojmeena0001/job_skills_extractor/blob/main/preprocessing.py"
    click SkillExtract "https://github.com/manojmeena0001/job_skills_extractor/blob/main/skill_extractor.py"
    click SkillCategorizer "https://github.com/manojmeena0001/job_skills_extractor/blob/main/skill_categorizer.py"
    click Utils "https://github.com/manojmeena0001/job_skills_extractor/blob/main/utils.py"
    click Dict "https://github.com/manojmeena0001/job_skills_extractor/blob/main/data/skill_dictionary.json"
    click Categories "https://github.com/manojmeena0001/job_skills_extractor/blob/main/data/skill_categories.json"
    click Assets "https://github.com/manojmeena0001/job_skills_extractor/tree/main/attached_assets/"

    %% Styles
    classDef ui fill:#D0E8FF,stroke:#0066CC,color:#003366
    classDef proc fill:#DFF2D8,stroke:#4F8A10,color:#33691E
    classDef data fill:#FFDDB3,stroke:#D48806,color:#663C00
    classDef ext fill:#E0E0E0,stroke:#888888,color:#555555
    classDef shared fill:#FFF4E5,stroke:#CC9900,color:#996600
    classDef assets fill:#FFE5F0,stroke:#CC3366,color:#990040
```
