# sfrfl-17304/dxc-internship architecture

> Architecture diagram of the GitHub repository sfrfl-17304/dxc-internship, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-05). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/sfrfl-17304/dxc-internship
- Repository: https://github.com/sfrfl-17304/dxc-internship

## Overview

##### Project Type and Purpose
- This is a full-stack, data-driven web application (“Hybrid Search Engine for HR”) that combines:
- A Streamlit frontend (app.py)
- A Python/LangChain backend pipeline (cv_processor, loaders)
- MongoDB for storage and filtering
- External AI services (Groq Llama3 for extraction, Ollama mxbai-embed-large for embeddings)

##### File Structure
Top-level directories and files:
- app.py – Streamlit application entry point (frontend + backend glue)
- docker-compose.yml – Brings up MongoDB and Mongo Express
- hybrid_search_mongodb/database_setup/setup_database.py – DB initialization scripts
- hybrid_search_mongodb/hr_engine/cv_processor.py – Core CV parsing and metadata extraction
- loaders/loaders.py – Document ingestion pipeline (file → text)
- source_documents/ – Raw CV files (PDF/DOCX)
- requirements.txt, .env – Dependencies and config

##### README for Additional Insights
- Confirms the architectural components and emphasizes:
- Hybrid search pattern: structured filtering (MongoDB queries) + semantic ranking (vector store)
- Manual override filters (UI elements)
- Dockerized database layer, local/remote AI services
- Future expansion points: REST APIs, analytics, multi-language

##### Main Components & Interactions
a. Frontend (Streamlit)
– Handles UI: file upload, filter controls, natural language search input, results display
– Runs in the user’s browser; back-and-forth via HTTP/WebSocket to app.py
b. Backend (app.py + LangChain pipelines)
– Routes user actions:
- “Process CVs” → loaders → cv_processor → embedding/extraction services → MongoDB
- “Search” → builds a MongoDB filter query + retrieves vector embedding of query → calls MongoDB Atlas (or local) + nearest-neighbor vector search → merges and ranks results
c. Document Store (MongoDB + Mongo Express)
– Stores structured CV records (name, skills, experience)
– Holds vector embeddings in a collection (with an index for ANN search)
d. External AI Services
– Groq Llama3-70B (hosted via API) for extracting structured fields from raw text
– Ollama local service (mxbai-embed-large) for computing embedding vectors
e. Orchestration/Deployment
– Docker Compose spins up:
- MongoDB (port 27017)
- Mongo Express (port 8081)
– .env provides credentials

##### Architectural Patterns & Design Principles
- Hybrid Search Pattern: Combine database filtering + vector search
- Pipeline/Batched Processing: Separate ingestion (loaders) from processing (cv_processor)
- Separation of Concerns: UI, business logic, data store, AI services
- Configurability: Use .env for keys, models, endpoints
- Extensibility: Modular processor code, easy to swap embedding or extraction models

## Diagram (Mermaid)

```mermaid
graph TD
    %% UI Layer
    subgraph "UI Layer"
        UI["Streamlit UI"]:::ui
    end

    %% Application Layer
    subgraph "Application Layer"
        APP["app.py"]:::app
        LOAD["Loaders"]:::app
        CVP["CV Processor"]:::app
        ENV[".env"]:::config
        REQ["requirements.txt"]:::config
    end

    %% Data Layer
    subgraph "Data Layer"
        MONGO["MongoDB"]:::db
        MEXP["Mongo Express"]:::db
        SETUP["DB Init"]:::app
        SRC["source_documents/"]:::data
    end

    %% External AI Services
    subgraph "External AI Services"
        GROQ["Groq Llama3-70B"]:::external
        OLLAMA["Ollama mxbai-embed-large"]:::external
    end

    %% Orchestration
    subgraph "Orchestration"
        DOCKER["docker-compose.yml"]:::config
    end

    %% Documentation & Diagrams
    subgraph "Docs & Diagrams"
        README["README.md"]:::docs
        DIAG["HybridSearch.drawio"]:::docs
    end

    %% Data Flows
    User([User]):::ui -->|uses UI| UI
    UI -->|"POST /process"| APP
    UI -->|"GET /search"| APP

    APP -->|"load files"| SRC
    SRC -->|"input to pipeline"| LOAD
    LOAD -->|"invoke parsing"| CVP
    CVP -->|"Extract metadata via Groq API"| GROQ
    CVP -->|"Compute embeddings via Ollama"| OLLAMA
    CVP -->|"Store metadata & vectors"| MONGO

    APP -->|"Structured query"| MONGO
    APP -->|"Vector ANN search"| MONGO
    MONGO -->|"Results"| APP
    APP -->|"responses"| UI

    ENV --> DOCKER
    DOCKER --> MONGO
    DOCKER --> MEXP
    SETUP --> MONGO

    %% References
    README --> APP
    README --> DOCKER
    README --> SETUP
    README --> CVP
    README --> LOAD
    README --> SRC
    README --> REQ
    README --> DIAG

    %% Click Events
    click APP "https://github.com/sfrfl-17304/dxc-internship/blob/master/app.py"
    click DOCKER "https://github.com/sfrfl-17304/dxc-internship/blob/master/docker-compose.yml"
    click SETUP "https://github.com/sfrfl-17304/dxc-internship/blob/master/hybrid_search_mongodb/database_setup/setup_database.py"
    click CVP "https://github.com/sfrfl-17304/dxc-internship/blob/master/hybrid_search_mongodb/hr_engine/cv_processor.py"
    click LOAD "https://github.com/sfrfl-17304/dxc-internship/blob/master/loaders/loaders.py"
    click SRC "https://github.com/sfrfl-17304/dxc-internship/tree/master/source_documents/"
    click REQ "https://github.com/sfrfl-17304/dxc-internship/blob/master/requirements.txt"
    click DIAG "https://github.com/sfrfl-17304/dxc-internship/blob/master/Diagrams/HybridSearch.drawio"
    click README "https://github.com/sfrfl-17304/dxc-internship/blob/master/README.md"

    %% Styles
    classDef ui fill:#D4E4F7,stroke:#0366d6,stroke-width:2px
    classDef app fill:#D6F5D6,stroke:#28A745,stroke-width:2px
    classDef db fill:#FFE5B4,stroke:#E07A5F,stroke-width:2px
    classDef external fill:#E0E0E0,stroke:#555555,stroke-width:2px
    classDef config fill:#F0D9FF,stroke:#9B59B6,stroke-width:2px
    classDef data fill:#FFF3B0,stroke:#F4A261,stroke-width:2px
    classDef docs fill:#F5F5F5,stroke:#CCCCCC,stroke-width:2px
```
