# bhavani458/vestaequity_qa architecture

> Architecture diagram of the GitHub repository bhavani458/vestaequity_qa, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-03-31). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/bhavani458/vestaequity_qa
- Repository: https://github.com/bhavani458/vestaequity_qa

## Overview

##### Project Type and Purpose
- The project is a specialized question-answering assistant for a FAQ page.
- It uses a combination of natural language processing (via OpenAI’s LLM), embeddings (with HuggingFace’s InstructorEmbedding), and a FAISS vector database to provide quick, accurate responses.
- A lightweight Streamlit interface is used for user interaction.
- The overall design follows a full-stack-like pipeline where data is ingested, processed with vector embeddings, stored/retrieved from a vector database, and results are generated via an LLM.

##### File Structure
- Top-Level Files:
- README.md and LICENSE give high-level project information.
- requirements.txt specifies dependencies.
- main.py is likely the main entry point.
- langchain_helper.py likely encapsulates helper functions for working with LangChain – such as loading data, processing embeddings, and querying the vector DB.
- faqs.csv is the data source.
- Directories:
- faiss_index holds the FAISS index files (index.faiss and index.pkl); these files show the vector database storage.
- __pycache__ is present, but not essential for design.

##### Main Components
a. Data Ingestion Component:
- Data Source: faqs.csv
- Responsible for loading the FAQ data.
b. Data Processing and Embeddings:
- The LangChain helper (in langchain_helper.py) wraps the CSVLoader functionality.
- Embeddings are generated using the InstructorEmbedding model.
c. FAISS Vector Database:
- Responsible for storing the vectorized FAQ data.
- Uses files located in the faiss_index directory (index.faiss, index.pkl).
d. Query/Response Generator:
- Query phase: The system receives a question from the user.
- It uses similarity search (cosine similarity) on the vector database to fetch related FAQ entries.
- OpenAI’s API (LLM) is then used to generate a detailed, contextual answer.
e. User Interface:
- A Streamlit-based web interface (either in app.py or main.py) for user interactions.
- Provides buttons for processing FAQs and input fields for user queries.

##### Relationships and Data Flow
- The flow starts when a user enters a query into the Streamlit interface.
- The system loads the faqs.csv (either at startup or on-demand).
- The LangChain helper processes the CSV data to create embeddings with the InstructorEmbedding model.
- These embeddings are stored or updated into the FAISS vector database located in the faiss_index folder.
- When a query is received, a similarity search in the FAISS vector index retrieves the most relevant FAQ entries.
- The retrieved data is then fed as context to OpenAI’s LLM, which generates the answer.
- The answer is displayed back to the user via the Streamlit UI.
- UI → Data Processing (indicating user query starting the process).
- Data Processing → FAISS (storing and retrieving embedded vectors).
- FAISS → Query Engine (sending search results).
- Query Engine → OpenAI LLM (to generate final responses).
- OpenAI LLM → UI (sending back final answer).

##### Architectural Patterns/Design Principles
- Clear separation of concerns: Data handling (CSV ingestion), processing (embeddings), storage (FAISS), business logic (query resolution), and presentation (Streamlit UI).
- Pipeline pattern: It shows a step-by-step flow from loading FAQs to generating answers.
- Modular approach: With a dedicated helper module (langchain_helper.py) for LangChain integration and separate areas for indexing and UI.

## Diagram (Mermaid)

```mermaid
flowchart TD
    subgraph "Data Ingestion Pipeline"
        A["FAQ Data"]:::data
        B["LangChain Helper"]:::process
        C["InstructorEmbedding (HuggingFace)"]:::process
        D["FAISS Vector Database"]:::storage
    end

    subgraph "Query Flow"
        E["Streamlit Interface"]:::ui
        F["Similarity Search (Cosine)"]:::process
        G["OpenAI API"]:::external
    end

    A -->|"load_faqs"| B
    B -->|"process_data"| C
    C -->|"generate_embeddings"| D

    E -->|"submit_query"| F
    F -->|"search_vectors"| D
    D -->|"return_results"| F
    F -->|"call_OpenAI"| G
    G -->|"send_answer"| E

    click A "https://github.com/bhavani458/vestaequity_qa/blob/main/faqs.csv"
    click B "https://github.com/bhavani458/vestaequity_qa/blob/main/langchain_helper.py"
    click D "https://github.com/bhavani458/vestaequity_qa/tree/main/faiss_index"
    click E "https://github.com/bhavani458/vestaequity_qa/blob/main/main.py"

    classDef data fill:#BBDEFB,stroke:#0D47A1,stroke-width:2px
    classDef process fill:#C8E6C9,stroke:#388E3C,stroke-width:2px
    classDef storage fill:#BBDEFB,stroke:#1976D2,stroke-width:2px
    classDef ui fill:#FFCC80,stroke:#F57C00,stroke-width:2px
    classDef external fill:#FFF59D,stroke:#FBC02D,stroke-width:2px
```
