# muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system architecture

> Architecture diagram of the GitHub repository muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-26). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system
- Repository: https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system

## Overview

##### Project Type and Purpose
- This is a Python-based Retrieval-Augmented Generation (RAG) system with a Streamlit UI.
- Its goal is to let users (in Arabic or English) query a heterogeneous knowledge graph built from a corpus of PDF documents (Saudi Food & Drug Authority specifications), retrieve relevant passages via combined vector & graph heuristics, and generate concise, citation-free answers.

##### File Structure
- Top level:
- data/ – raw artifacts: FAISS index, pickled graph mappings, knowledge graphs
- rags/ – source PDF documents
- src/ – core modules: loader, indexer, retriever, generator, config, main entrypoint, model list
- streamlit_app.py – front-end application
- requirements.txt – dependencies (FAISS, spaCy, LLM client, Streamlit)
- src/ components indicate a classic pipeline:
- loader.py reads & preprocesses PDFs into text & entities
- graph_indexer.py builds/updates the heterogeneous knowledge graph
- retriever.py performs combined semantic + graph-heuristic retrieval against FAISS index
- generator.py stitches retrieved docs into prompts and calls LLM(s)
- list_models.py enumerates available LLM models
- config.py centralizes settings (paths, model names, language options)
- main.py orchestrates CLI runs (batch indexing, retrieval)
- streamlit_app.py provides the interactive UI with language switcher and graph visualization.

##### README for Additional Insights
- Bilingual support via spaCy models en_core_web_sm & xx_ent_wiki_sm
- FAISS directory holds prebuilt semantic.index and doc_mapping.pkl
- Knowledge graphs serialized as .pkl (bakery_graph, saudia_drugs_graph, sfda_knowledge_graph)
- System merges miniRAG–style graph retrieval with coRAG–style generation

a. Main Components
- User Interface: Streamlit front end
- Document Repository: PDFs under rags/
- Loader & Preprocessor: Text extraction + entity recognition (spaCy)
- Graph Indexer: Builds heterogeneous knowledge graph (NetworkX or similar)
- Vector Store: FAISS index for semantic search
- Retriever: Hybrid semantic + graph-based retrieval logic
- Generator: Prompts an LLM for answer generation (OpenAI, HuggingFace)
- Configuration Module: Central configurables (paths, model names)
- Model Directory: list_models component
- Knowledge Graph Storage: Serialized .pkl graph files
b. Relationships & Data Flow
1. User → Streamlit UI → Query (text + language choice)
4. UI displays answer and optional graph visualization
5. Offline/batch: loader → graph_indexer → updates FAISS & graph storage
c. Architectural Patterns & Principles
- Pipeline pattern: loader → indexer → retriever → generator
- Separation of concerns: distinct modules for data ingestion, indexing, retrieval, and generation
- Hybrid retrieval: semantic vectors + explicit graph heuristics for cross-document connections
- Pluggable LLM backends via list_models.py and config.py
d. Technologies, Frameworks, Libraries
- Python 3.10
- Streamlit (UI)
- spaCy (NER & multilingual tokenization)
- FAISS (vector database)
- NetworkX or similar for graph construction (implied)
- OpenAI or HuggingFace LLM client
- pickle for serialization

##### Detailed Capture of Architectural Elements
- Under “Generator,” note prompt assembly logic and citation-free policy.
- In the UI block, add a toggle for Arabic/English and a panel for graph visualization.

## Diagram (Mermaid)

```mermaid
flowchart TB
    %% User and UI
    User(["User"]):::user
    UI["Streamlit UI\n(language toggle, graph viz)"]:::module
    click UI "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/streamlit_app.py"
    User --> UI

    %% Configuration and Model List
    Config["Configuration Module"]:::module
    click Config "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/src/config.py"
    ListModels["Model Directory\n(list_models)"]:::module
    click ListModels "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/src/list_models.py"
    Config --> Loader
    Config --> GraphIndexer
    Config --> RetrieverComp
    Config --> Generator
    ListModels --> Generator

    %% Offline Pipeline
    subgraph "Offline Pipeline"
        Docs["PDF Documents\n(rags/)"]:::datastore
        click Docs "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/tree/main/rags/"
        Loader["Loader & Preprocessor"]:::module
        click Loader "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/src/loader.py"
        GraphIndexer["Graph Indexer"]:::module
        click GraphIndexer "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/src/graph_indexer.py"
        Docs --> Loader --> GraphIndexer
        GraphIndexer --> FAISSIndex
        GraphIndexer --> KG1
        GraphIndexer --> KG2
        GraphIndexer --> KG3
    end

    %% FAISS Data Stores
    FAISSIndex["FAISS Index"]:::datastore
    click FAISSIndex "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/data/faiss/semantic.index"
    DocMapping["Doc Mapping"]:::datastore
    click DocMapping "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/data/faiss/doc_mapping.pkl"

    %% Knowledge Graph Storage
    subgraph "Knowledge Graph Storage"
        KG1["bakery_graph.pkl"]:::datastore
        click KG1 "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/data/graphs/bakery_graph.pkl"
        KG2["saudia_drugs_graph.pkl"]:::datastore
        click KG2 "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/data/graphs/saudia_drugs_graph.pkl"
        KG3["sfda_knowledge_graph.pkl"]:::datastore
        click KG3 "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/data/graphs/sfda_knowledge_graph.pkl"
    end

    %% Retrieval Pipeline
    UI --> RetrieverComp["Retriever"]:::module
    subgraph Retriever
        SS["Semantic Search"]:::module
        GH["Graph Heuristics"]:::module
    end
    click SS "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/src/retriever.py"
    click GH "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/src/retriever.py"
    RetrieverComp --> SS
    RetrieverComp --> GH
    SS --> FAISSIndex
    SS --> DocMapping
    GH --> KG1
    GH --> KG2
    GH --> KG3
    SS --> Retrieved
    GH --> Retrieved

    Retrieved["Retrieved Context\n(passages + subgraph)"]:::module
    Retrieved --> Generator

    %% Generation Pipeline
    Generator["Generator\n(prompt assembly & LLM)"]:::module
    click Generator "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/src/generator.py"
    Generator --> LLM
    LLM["LLM API"]:::external
    LLM --> Generator
    Generator --> UI

    %% Main CLI Entrypoint
    Main["Main Entrypoint\n(CLI)"]:::module
    click Main "https://github.com/muhammadahmedyaqoob/saudi-food-and-drug-authority-rag-system/blob/main/src/main.py"
    Main --> Loader
    Main --> GraphIndexer

    %% Styles
    classDef module fill:#ADD8E6,stroke:#000,stroke-width:1px
    classDef datastore fill:#90EE90,stroke:#000,stroke-width:1px
    classDef external fill:#FFA500,stroke:#000,stroke-width:1px
    classDef user fill:#D3D3D3,stroke:#000,stroke-width:1px
```
