How naman1995jain/shl-assessment-recommendation-system works
Main Components
• User Interfaces:
• Web UI: Represented by app.py (Streamlit application) that HR professionals or recruiters interact with.
• API Endpoint: Represented by api.py (FastAPI) which allows programmatic access to the system.
• Data Acquisition & Preparation:
• Scraping Component: scrape_shl2.py is responsible for gathering SHL assessment data, which is cached in shl_assessments.csv.
• Data Storage: The CSV file (shl_assessments.csv) acts as a local data store or cache.
• Processing & Integration:
• OpenRouter API Integration: openrouter_api.py handles interactions with the OpenRouter API (using the Mistral model) to generate embeddings and perform natural language content generation.
• Data Processing: Pandas and NumPy are used for data manipulation, while Plotly handles data visualization.
• Configuration & Environment:
• .env file: Stores essential API keys (e.g., the one for OpenRouter).
Out the Relationships and Data Flow
• The user (via web UI or API client) sends a query or job description.
• The Streamlit app (app.py) and/or FastAPI (api.py) receive the query.
• These components invoke the OpenRouter API (through openrouter_api.py) to transform and understand the semantics of the query, using AI embeddings.
• Data from shl_assessments.csv (populated by scrape_shl2.py) is processed using data libraries (Pandas/NumPy) and filtered based on the semantic match.
• The system then generates recommendations, including assessments metadata and candidate explanations for relevance.
• If using the Streamlit interface, the recommendations are further visualized using Plotly.
• The API endpoint returns a structured JSON response to the caller.
Architectural Patterns and Design Principles
• Separation of Concerns: The architecture clearly separates the user presentation layer (Streamlit UI), the API layer (FastAPI), and backend processing/integration (scraping, data processing, external API calls).
• Dual Interfaces: Recognize the design decision to support both interactive (web) access and programmatic (API) usage.
• External Service Integration: Emphasize the reliance on an external service (OpenRouter API) for advanced natural language processing tasks.
• Data Caching: The design uses persistent CSV storage for assessment data, ensuring that even if scraping isn’t run continuously, cached data is available.
Final Advice
• Every component is clearly labeled (e.g., “Streamlit Web UI (app.py)”, “FastAPI Service (api.py)”, “OpenRouter Integration (openrouter_api.py)”, “Data Scraper (scrape_shl2.py)”).