How sandipkatel/unified-infosec-qna-assistant works
Project Type and Purpose
• This is a full-stack Retrieval-Augmented Generation (RAG) AI system called “Unified InfoSec QnA Assistant.”
• Frontend: Next.js (TypeScript, Tailwind, React Query) for multi-page UI (batch upload, chat).
• Backend: Django REST API delivering endpoints that drive FAISS-based vector search + LLM invocation (Ollama via LangChain).
• Knowledge Base: CSV Q&A pairs + PDF policy docs indexed by FAISS.
File-Structure Analysis
• Top level:
• Backend/ – Django project and virtual-env requirements
– Backend/Backend/: Django settings, URLs, ASGI/WGI, views, utils (chat_data, history, main).
– faiss_csv_index/, faiss_pdf_index/: pre-built FAISS indexes (.faiss + .pkl).
• Frontend/ – Next.js project
– src/app/: page.js, layout.js, multiple pages (page1.js, page3.js).
– src/components/: chatHistory.js, policyDocumentList.js.
– public/: static PDF policy files.
• README: confirms Tech Stack, features (batch & conversational modes), vector search + LLM details, startup instructions.
Key Components
a. User Interface (Next.js)
• Pages: Batch Upload, Chat Interface
• Components: Policy List, Chat History
b. API Layer (Django REST)
• Endpoints: /upload-questionnaire, /chat
• Business Logic: uses utils/history.py, utils/main.py for context management
c. Vector Store (FAISS)
• CSV Index service (faiss_csv_index)
• PDF Index service (faiss_pdf_index)
d. LLM Service (Ollama via LangChain)
• Chat & prompt orchestration
e. Knowledge Base Storage
• Static CSV files (in Backend/utils/chat_data/threads)
• PDF policies (public/)
• User metadata (users.json)
Relationships and Data Flow
1. User → Next.js UI (select upload or chat)
2. Next.js → Django REST API
3. Django API
• loads user/chat history → utils/history.py
• performs hybrid retrieval:
– SentenceTransformers embedding
– FAISS lookup (CSV and/or PDF index)
• builds prompt with citations
• calls LLM (Ollama via LangChain)
4. LLM → Django returns answer + metadata
5. Django → Next.js UI (renders response with citations)
Architectural Patterns & Principles
• Separation of concerns: UI, API logic, retrieval, LLM integration.
• Monolithic backend with service-style modules (utils, faiss index).
• Hybrid semantic search + symbolic retrieval.
• Stateless API layer; external local services (Ollama) and file-based vector stores.
• Step 6: Show static file store (CSV files, PDF policy docs) feeding into “Vector Store.”