How arnnv/shl-recommendation-system works
Project type and purpose
• This is a full-stack, microservice-style recommendation system (“SHL Assessment Recommender”) that uses a FastAPI backend to serve recommendations and a Streamlit frontend for user interaction.
• It includes an auxiliary crawler service to ingest SHL assessment data, a hybrid search engine (vector + BM25), and an LLM-powered orchestration layer (LangGraph + LangChain).
File structure
• Top-level:
• app.py (FastAPI entry point)
• main.py (core LangGraph workflow / recommendation engine)
• crawler/ (crawler.py plus JSON data and state files)
• frontend/ (Streamlit app, requirements, Dockerfile)
• Dockerfile, .env.example, requirements.txt (backend Docker & dependencies)
• Patterns:
• Microservices: separate backend and frontend directories with their own Dockerfiles
• Clear separation of concerns: crawler service, recommendation logic, API layer, UI layer
• Deployment: Dockerfile and Procfile indicate containerized deployment; .env for secrets
README for additional insights
• Hybrid retrieval combining FAISS-via-Qdrant and BM25
• LangGraph orchestrates multi-step flows: parse job description → retrieve → rerank → respond
• External dependencies: Qdrant vector DB, OpenAI Embeddings, Anthropic / Claude LLMs
• Streaming: Uvicorn as ASGI server for FastAPI, Streamlit for UI
Detail
• Every major file/service folder as a component (crawler, frontend, backend, data stores)
• Docker and env var configuration as deployment details if needed
• Don’t over-abstract; show each moving part and its interactions to reflect the real architecture accurately