How stellarphantasm30/hack_the_future_hackathon works
Project Type & Purpose
• This is a _data‑processing & AI‑driven pipeline_ with a separate visualization frontend (Streamlit).
• It ingests PDF resumes and CSV job records, runs multi‑agent LLM/OCR processing, builds a vector index for semantic matching, generates outreach emails, persists everything in SQLite, and visualizes via Streamlit.
File Structure
• Top‑level scripts represent discrete pipeline stages:
• job_data_extraction.py
• job_summary_extraction.py
• document_processing.py
• candidate_pii_extraction.py
• resume_summary_extraction.py
• resume_vector_db.py
• resume_matching.py
• email_templating.py
• 01_DashBoard.py (Streamlit UI)
• “exported_data/” holds sample Excel exports from the SQLite DB.
• requirements.txt, .gitignore, assets/ (UI assets), README.md.
Key Architecture From README
• External services: Ollama (LLM), Tesseract (OCR), Faiss (HNSW)
• Data stores:
• SQLite3 (candidates.db) for structured tables
• Faiss index files for vector embeddings
• Tech Stack: Python, Pandas, Streamlit
Main Components & Their Relationships
1. Data Ingestion
• Input: PDF resumes, CSV of job titles/descriptions
• Script: job_data_extraction.py → writes raw jobs to SQLite
2. Job Analysis
• Script: job_summary_extraction.py
• Calls: Ollama
• Writes: job key‑points back to SQLite
3. Resume OCR & Raw Text
• Script: document_processing.py
• Calls: Tesseract via Docling
• Writes: raw text path/content into SQLite
4. PII Extraction
• Script: candidate_pii_extraction.py
• Calls: Ollama
• Writes: name, email, phone to SQLite
5. Resume Key‑Info Extraction
• Script: resume_summary_extraction.py
• Calls: Ollama
• Writes: skills, summaries to SQLite
6. Vectorization & Indexing
• Script: resume_vector_db.py
• Calls: Faiss (HNSW)
• Outputs: vector index files + mapping into SQLite
7. Matching & Scoring
• Script: resume_matching.py
• Uses: vector index + job key‑points
• Calls: Ollama for detailed justifications
• Writes: top matches + scores into SQLite
8. Email Generation
• Script: email_templating.py
• Calls: Ollama
• Writes: personalized draft emails into SQLite
9. Visualization
• Script: 01_DashBoard.py (Streamlit)
• Reads: SQLite + Faiss index (if needed)
• Displays: jobs, candidates, match details, emails
Architectural Patterns & Principles
• Pipeline / Batch Orchestration: discrete, idempotent stages
• Separation of Concerns: each Python script is a single-purpose agent
• Data‑Centric Design: SQLite as single source of truth, Faiss for vector store