# pallavi2908/semantra architecture

> Architecture diagram of the GitHub repository pallavi2908/semantra, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-06-24). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/pallavi2908/semantra
- Repository: https://github.com/pallavi2908/semantra

## Overview

##### Project Type and Purpose
- Semantra is a full-stack AI-powered medical‐query assistant.
- It consists of a static frontend (HTML/CSS/JS) served by a FastAPI backend on Uvicorn, which handles queries by retrieving semantically relevant chunks from a Qdrant vector store and invoking an LLM (Mistral 7B) for response generation.

##### File Structure
- Top-level scripts (create_chunks.py, convert_nested_to_flat.py, embeddings.py, merge_embeddings.py, importVectors.py, upload_to_qdrant.py) implement the offline data-ingestion and indexing pipeline.
- api.py / main.py define the FastAPI app and HTTP endpoints.
- qdrant.py encapsulates interactions with the Qdrant vector database.
- static/ holds the HTML/CSS frontend, which makes POST requests to `/query`.
- GitHub Actions (/.github/workflows/main.yml) pings the server to prevent idling on Render.

##### Architecture from the README
- Backend: FastAPI + Uvicorn, Qdrant vector DB replacing Weaviate, Mistral 7B LLM.
- Frontend: Static HTML/CSS with JavaScript fetch calls to `/query`.
- Deployment: Hosted on Render; CI/CD pinger via GitHub Actions.

##### Main Components & Interactions
a. Components
- Client Browser (User Interface)
- Static Assets (index.html, styles.css, JS)
- FastAPI Server (api.py)
- Qdrant Vector Database
- LLM Inference Engine (Mistral 7B)
- Data-Ingestion Pipeline (Python scripts + raw research papers)
- CI/CD Keep-Alive Service (GitHub Actions)
- Cloud Hosting (Render)
b. Relationships & Data Flow
1. User enters a query in the browser → JS POSTs to FastAPI `/query` endpoint.
2. FastAPI receives the query → calls qdrant.py to perform a vector search over embeddings stored in Qdrant.
3. Retrieved chunks + original query → sent to the LLM inference engine for answer generation.
4. LLM response → formatted by FastAPI → returned to the client.
5. Offline Pipeline: Raw documents → chunking → embedding generation → merged and imported into Qdrant.
6. GitHub Actions periodically pings the FastAPI endpoint to keep Render’s free instance alive.
c. Architectural Patterns & Principles
- Separation of concerns: static frontend vs. API vs. vector store vs. LLM.
- Asynchronous, non-blocking I/O via Uvicorn/ASGI.
- Modular ingestion pipeline for data updates.
d. Key Technologies
- FastAPI, Uvicorn, Qdrant, Mistral 7B, Python (requests, pandas, scikit-learn), Render, GitHub Actions.

## Diagram (Mermaid)

```mermaid
graph TB
    classDef frontend fill:#D0E8FF,stroke:#0066CC,color:#003366
    classDef backend fill:#D0FFCC,stroke:#009933,color:#004d1a
    classDef database fill:#FFE0B3,stroke:#CC8500,color:#663D00
    classDef llm fill:#FFF2CC,stroke:#CC9900,color:#666600,shape:hexagon
    classDef batch fill:#E0E0E0,stroke:#888888,color:#333333
    classDef devops fill:#F3E6FF,stroke:#9933CC,color:#4D0052

    subgraph "Frontend Layer"
        Browser["Client Browser"]:::frontend
        subgraph "Static Assets"
            StaticDir["static/"]:::frontend
            Index["index.html"]:::frontend
            Styles["styles.css"]:::frontend
        end
    end

    subgraph "Backend Layer"
        subgraph "API Server"
            API_py["api.py"]:::backend
            Main_py["main.py"]:::backend
        end
        QConnector["qdrant.py"]:::backend
        QdrantDB["Qdrant Vector Database"]:::database
        LLME["LLM Inference Engine (Mistral 7B)"]:::llm
    end

    Hosting["Render Hosting"]:::devops
    CIDP["CI/CD Keep-Alive (GitHub Actions)"]:::devops
    Requirements["requirements.txt"]:::devops

    subgraph "Batch Processing Pipeline"
        RawDocs["Raw Documents"]:::batch
        subgraph "Chunking"
            Chunk1["create_chunks.py"]:::batch
            Chunk2["convert_nested_to_flat.py"]:::batch
        end
        subgraph "Embedding Generation"
            Emb1["embeddings.py"]:::batch
            Emb2["merge_embeddings.py"]:::batch
        end
        subgraph "Vector Import"
            Imp1["importVectors.py"]:::batch
            Imp2["upload_to_qdrant.py"]:::batch
        end
    end

    Browser -->|loads| Index
    Browser -->|loads| Styles
    Index -->|POST /query| API_py
    Main_py -->|defines FastAPI app| API_py
    API_py -->|"calls connector"| QConnector
    QConnector -->|"vector search"| QdrantDB
    QConnector -->|returns chunks| API_py
    API_py -->|"invoke"| LLME
    LLME -->|"response"| API_py
    API_py -->|JSON response| Index
    CIDP -->|periodic ping| API_py
    Hosting -->|hosts| API_py
    Requirements -->|dependencies| API_py

    RawDocs --> Chunk1
    Chunk1 --> Chunk2
    Chunk2 --> Emb1
    Emb1 --> Emb2
    Emb2 --> Imp1
    Imp1 --> Imp2
    Imp2 --> QdrantDB

    click Index "https://github.com/pallavi2908/semantra/blob/main/static/index.html"
    click Styles "https://github.com/pallavi2908/semantra/blob/main/static/styles.css"
    click StaticDir "https://github.com/pallavi2908/semantra/tree/main/static/"
    click API_py "https://github.com/pallavi2908/semantra/blob/main/api.py"
    click Main_py "https://github.com/pallavi2908/semantra/blob/main/main.py"
    click QConnector "https://github.com/pallavi2908/semantra/blob/main/qdrant.py"
    click Chunk1 "https://github.com/pallavi2908/semantra/blob/main/create_chunks.py"
    click Chunk2 "https://github.com/pallavi2908/semantra/blob/main/convert_nested_to_flat.py"
    click Emb1 "https://github.com/pallavi2908/semantra/blob/main/embeddings.py"
    click Emb2 "https://github.com/pallavi2908/semantra/blob/main/merge_embeddings.py"
    click Imp1 "https://github.com/pallavi2908/semantra/blob/main/importVectors.py"
    click Imp2 "https://github.com/pallavi2908/semantra/blob/main/upload_to_qdrant.py"
    click CIDP "https://github.com/pallavi2908/semantra/blob/main/.github/workflows/main.yml"
    click Requirements "https://github.com/pallavi2908/semantra/blob/main/requirements.txt"
```
