How tarunima-ds/rbi_ai_policy_assistant works
This repository is an RBI-document question-answering assistant with a separate PDF indexing workflow. An operator can index a PDF from the documents folder; ingestion extracts pages, prepares paragraphs and chunks, creates embeddings, and saves a FAISS index. A user asks a question in the Streamlit chat, which retrieves relevant chunks and sends them with the question to Gemini for an answer. The source index also identifies text-cleaning and metadata helpers, but their relationship to the main ingestion path is not shown. The README is empty, so unsampled capabilities and wiring are not inferred.
Main entry points: ingestion.py Ingestion pipeline · app.py Indexing entry · text_extractor.py Text extractor · rag.py RAG pipeline
Components 22
Document ingestion
- Indexing entry
app.py - Ingestion pipeline
pipeline/ingestion.py - Logger setup
utils/logger.py
Text processing
- PDF loader
services/pdf_loader.py - File validator
services/file_validator.py - Paragraph processor
services/paragraph_processor.py - Paragraph detector
services/paragraph_detector.py - Chunker
services/chunker.py - Embedding generator
services/embedding.py - Text extractor
services/text_extractor.py - Text cleaner
services/cleaner.py - Header footer remover
services/header_footer.py - Metadata extractor
services/metadata.py
Retrieval and generation
- FAISS vector store
services/vectorstore.py - Retriever
services/retriever.py - RAG pipeline
pipeline/rag.py - Gemini answer service
services/llm.py
User interface
- Streamlit chat
dashboard/app.py
Other components
- Operator
- RBI PDF corpus
documents - Gemini API
- Question asker
Connections 21
- Operator → Indexing entry: runs indexing
- Indexing entry → RBI PDF corpus: selects PDFs
app.py - Indexing entry → Ingestion pipeline: starts pipeline
app.py - Indexing entry → Logger setup: configures logging
app.py - Ingestion pipeline → PDF loader: extracts pages
pipeline/ingestion.py - PDF loader → File validator: validates file
services/pdf_loader.py - Ingestion pipeline → Paragraph processor: processes pages
pipeline/ingestion.py - Paragraph processor → Paragraph detector: splits paragraphs
services/paragraph_processor.py - Ingestion pipeline → Chunker: chunks paragraphs
pipeline/ingestion.py - Ingestion pipeline → Embedding generator: embeds chunks
pipeline/ingestion.py - Ingestion pipeline → FAISS vector store: builds and saves
pipeline/ingestion.py - Text extractor → PDF loader: uses loader
services/text_extractor.py - Text extractor → Metadata extractor: uses metadata
services/text_extractor.py - Text extractor → Text cleaner: cleans text
services/text_extractor.py - Text cleaner → Header footer remover: uses remover
services/cleaner.py - Question asker → Streamlit chat: asks question
dashboard/app.py - Streamlit chat → RAG pipeline: calls ask
dashboard/app.py - RAG pipeline → Retriever: searches context
pipeline/rag.py - Retriever → FAISS vector store: searches index
services/retriever.py - RAG pipeline → Gemini answer service: requests answer
pipeline/rag.py - Gemini answer service → Gemini API: generates content
services/llm.py