# codex0870/nirf-college-ranking architecture

> Architecture diagram of the GitHub repository codex0870/nirf-college-ranking, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2026-02-21). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/codex0870/nirf-college-ranking
- Repository: https://github.com/codex0870/nirf-college-ranking

## Overview

##### What this project is (type + purpose)
This repo is primarily a **model packaging + inference usage** project, not a traditional multi-service application.

- **Core deliverable:** a **GGUF-converted, quantized Gemma 2B** model fine-tuned for **NIRF 2025 institutional ranking lookups**.
- **Primary runtime targets:** `llama.cpp`, `Ollama`, and `llama-cpp-python`.
- **Secondary/experimental workflows (from notebooks):**
- **Quantization / conversion experiments** (`gemma_quantization*.ipynb`)
- **RAG exploration** (`rag_1.ipynb`, `rag_2.ipynb`, `rag_3.ipynb`)
- **Web scraping** (`web_scrap.ipynb`)
- **Entry points / ops:**
- `use_gguf_model.sh` (shell helper to run the GGUF model)
- `main.py` (likely a python driver for inference or a small demo CLI)
- `requirements.txt` (python deps, especially if using llama-cpp-python / RAG tooling)

##### Offline model build/publish pipeline (how GGUF + quantization are produced)
2) **Online inference/usage architecture** (how a user query flows through runtimes like llama.cpp/Ollama/python, and optionally RAG)

##### How to read the file tree as architecture signals
From the top-level files:

- `README.md`: defines the **intended usage paths** (llama.cpp, Ollama, python).
- `gemma_quantization.ipynb`, `gemma_quantizationv2_0.ipynb`: imply an **offline pipeline stage** (convert → quantize → validate).
- `rag_*.ipynb`: indicates an optional **retrieval augmentation subsystem** (scrape/index/retrieve → augment prompt → LLM).
- `web_scrap.ipynb`: suggests a **data acquisition component** feeding RAG or dataset building.
- `main.py`: likely a lightweight “app” wrapper (CLI or script) around llama-cpp-python.
- `use_gguf_model.sh`: OS-level wrapper to run inference locally.

##### Source model (HuggingFace)
- Base: `google/gemma-2-2b-it`
- Fine-tuned model: `coderop12/gemma2b-nirf-lookup-2025` (QLoRA 4-bit)
##### Conversion tooling
- `llama.cpp` conversion scripts/tools (HuggingFace → GGUF F16)
##### Quantization step
- Quantize to `Q4_K_M` (recommended)
- `gemma_quantization*.ipynb` as the “orchestration / experimentation environment”
##### Published artifacts
- `*.gguf` files (F16 and Q4_K_M)
- Optionally “Modelfile” for Ollama import
##### Distribution
- HuggingFace repo / GitHub release / local artifact store (where users download from)

- HF Fine-tuned Model → (Convert) → GGUF F16 → (Quantize) → GGUF Q4_K_M → (Validate) → Publish

##### Online / Inference Usage Architecture (runtime-centric)

**Core inference paths (three “clients”):**
##### llama.cpp CLI path
- User → `llama-cli` → loads `*.gguf` → generates response
##### Ollama path
- User → `ollama run` → Ollama runtime → loads imported GGUF model → generates response
##### Python path (`llama-cpp-python`)
- User/App (`main.py`) → `llama_cpp.Llama(...)` → loads `*.gguf` → generates response

**Shared resources:**
- The **GGUF model file** on disk (artifact store)
- Compute resources: CPU/GPU (T4/RTX/etc.)

- User prompt → Runtime → Model → Token generation → Structured response

##### Optional RAG Subsystem (derived from `rag_*.ipynb` + `web_scrap.ipynb`)

Components:
##### Data acquisition
- `web_scrap.ipynb` → pulls NIRF pages / institutional info
##### Document store
- Raw HTML/text files (local disk / object store)
##### Embedding + indexing
- Embedding model (external dependency) → Vector DB / local FAISS index
##### Retriever
- Given user query → retrieve top-K docs/snippets
##### Prompt builder
- Combine user query + retrieved context + instruction template
##### LLM inference
- llama.cpp/Ollama/python runtime using GGUF model
##### Response
- Structured response with references

- Query → Retriever → Context → Prompt builder → LLM → Answer (+ citations)

- Actors: “User / Developer”
- External systems: HuggingFace, NIRF sources (if scraping), llama.cpp, Ollama
- System: “Gemma 2B NIRF Lookup GGUF package + scripts/notebooks”
- Artifacts: “GGUF model files”
- Containers: Notebooks (offline pipeline), Inference runtimes (llama.cpp / Ollama / Python app), Artifact storage, Optional RAG pipeline
- Documents → embeddings → retrieval → prompt assembly → generation

##### Relationships explicitly
- **Artifact boundaries**
- “Model artifact: `gemma2b...q4_k_m.gguf`” is a first-class node
- **Runtime boundaries**
- llama.cpp vs Ollama vs Python are separate execution environments
- **Compute/memory expectations**
- “~2GB VRAM Q4_K_M” / CPU option (a small annotation near runtimes)
- **Source of truth**
- “NIRF official sources” as external verification (even if not automated)

##### Make “optional” paths visually obvious
- RAG pipeline (since notebooks suggest exploration, not guaranteed production)
- Web scraping (optional data acquisition)

##### A “most accurate” component breakdown for this repo (don’t overthink; split it up)

**Offline**
- Fine-tuned HF model (QLoRA)
- Conversion step (HF → GGUF F16)
- Quantization step (F16 → Q4_K_M)
- Validation (NIRF queries)
- Published artifacts (GGUF files)

**Online**
- User
- Inference runtime (three alternatives: llama.cpp / Ollama / python)
- Local disk / artifact store containing GGUF
- Output formatter (structured response + references)

**Optional RAG**
- Web scraping notebook
- Raw corpus store
- Embedding model
- Vector index
- Retriever
- Prompt composer
- LLM inference (reuses online runtime)

## Diagram (Mermaid)

```mermaid
flowchart TD

%% =======================
%% Context / Overview
%% =======================
subgraph "Project Context (Model Packaging + Inference Usage)"
  direction TB

  User["User / Developer"]:::actor

  subgraph "Repository Contents (Scripts + Notebooks + Driver)"
    direction TB
    Readme["Usage Guide (llama.cpp / Ollama / python)"]:::doc
    MainPy["Python Driver (llama-cpp-python)"]:::runtime
    UseSh["Shell Helper (llama.cpp CLI)"]:::doc
    Reqs["Python Dependencies"]:::doc

    subgraph "Offline Notebooks (Build/Experiment)"
      direction TB
      QuantNB["Quantization / Conversion Notebook"]:::offline
      QuantNBv2["Quantization / Conversion Notebook (v2)"]:::offline
    end

    subgraph "Optional Notebooks (RAG + Scraping)"
      direction TB
      Rag1["RAG Notebook 1"]:::rag
      Rag2["RAG Notebook 2"]:::rag
      Rag3["RAG Notebook 3"]:::rag
      WebScrap["Web Scraping Notebook"]:::rag
    end
  end

  HFBase["HuggingFace: google/gemma-2-2b-it (Base)"]:::external
  HFFine["HuggingFace: coderop12/gemma2b-nirf-lookup-2025 (Fine-tuned QLoRA 4-bit)"]:::external
  NIRFSites["NIRF Official Sources (Web)"]:::external

  User -->|"reads_usage"| Readme
  User -->|"runs"| UseSh
  User -->|"runs"| MainPy
  User -->|"installs_deps"| Reqs

  QuantNB -->|"pulls_model"| HFFine
  QuantNBv2 -->|"pulls_model"| HFFine
  WebScrap -.->|"scrapes"| NIRFSites
end

%% =======================
%% Offline: Build & Release Pipeline
%% =======================
subgraph "Offline Model Build & Release Pipeline (Artifact-centric)"
  direction TB

  HFIn["Fine-tuned HF Model (QLoRA 4-bit)"]:::external
  Convert["Conversion Tooling (HF -> GGUF F16 via llama.cpp tools)"]:::offline
  GGUFf16(("GGUF Artifact (F16)")):::store
  Quantize["Quantization Step (F16 -> Q4_K_M)"]:::offline
  GGUFq4(("GGUF Artifact (Q4_K_M recommended)")):::store
  Validate["Functional Validation / Quality Checks (NIRF queries)"]:::offline
  Modelfile["Optional: Ollama Modelfile"]:::doc
  Publish["Distribution / Publish (HF repo / GitHub release / local artifacts)"]:::external

  HFIn -->|"convert"| Convert
  Convert -->|"produces"| GGUFf16
  GGUFf16 -->|"quantize"| Quantize
  Quantize -->|"produces"| GGUFq4
  GGUFq4 -->|"smoke_test"| Validate
  Validate -->|"publish"| Publish
  GGUFf16 -.->|"optional_publish"| Publish
  Modelfile -.->|"import_recipe"| Publish
end

%% Link notebooks to pipeline
QuantNB -->|"orchestrates"| Convert
QuantNB -->|"orchestrates"| Quantize
QuantNB -->|"runs_tests"| Validate
QuantNBv2 -->|"experiments_with"| Convert
QuantNBv2 -->|"experiments_with"| Quantize

HFFine -->|"is_source_of"| HFIn

%% =======================
%% Online: Inference Usage Architecture
%% =======================
subgraph "Online Inference Usage (Runtime-centric)"
  direction TB

  Prompt["User Prompt"]:::actor
  Disk(("Local Disk / Artifact Store (GGUF files)")):::store
  CPU["Compute: CPU / GPU (e.g., ~2GB VRAM for Q4_K_M)"]:::compute

  subgraph "Runtime Options (Choose One)"
    direction TB

    subgraph "A) llama.cpp Path"
      direction TB
      LlamaCLI["llama.cpp CLI (llama-cli)"]:::runtime
      ShHelper["use_gguf_model.sh (wrapper)"]:::doc
      ShHelper -->|"invokes"| LlamaCLI
    end

    subgraph "B) Ollama Path"
      direction TB
      Ollama["Ollama Runtime (ollama run)"]:::runtime
    end

    subgraph "C) Python Path"
      direction TB
      PyApp["main.py"]:::runtime
      LCPP["llama-cpp-python (llama_cpp.Llama)"]:::runtime
      PyApp -->|"calls"| LCPP
    end
  end

  Response["Generated Answer (Structured response + references)"]:::actor

  Prompt -->|"prompt"| LlamaCLI
  Prompt -->|"prompt"| Ollama
  Prompt -->|"prompt"| PyApp

  Disk -->|"loads_*.gguf"| LlamaCLI
  Disk -->|"loads_imported_gguf"| Ollama
  Disk -->|"loads_*.gguf"| LCPP

  CPU -->|"executes"| LlamaCLI
  CPU -->|"executes"| Ollama
  CPU -->|"executes"| LCPP

  LlamaCLI -->|"generated_tokens"| Response
  Ollama -->|"generated_tokens"| Response
  LCPP -->|"generated_tokens"| Response
end

%% Published artifacts flow to local disk
Publish -->|"download/install"| Disk

%% =======================
%% Optional: RAG Subsystem (Experimental)
%% =======================
subgraph "Optional RAG Subsystem (Notebook-driven, Experimental)"
  direction TB

  RAGQuery["User Query (RAG)"]:::actor

  Scrape["Data Acquisition (web_scrap.ipynb)"]:::rag
  RawStore(("Raw Documents Store (HTML/Text)")):::store
  Embed["Embedding Model (external dependency)"]:::external
  VecIdx(("Vector Index (FAISS / Vector DB)")):::store
  Retrieve["Retriever (Top-K)"]:::rag
  PromptBuild["Prompt Builder (Query + Context + Template)"]:::rag
  LLMReuse["LLM Inference (reuse any Online Runtime)"]:::runtime
  RAGResp["Answer + Citations/References"]:::actor

  Scrape -.->|"writes_docs"| RawStore
  RawStore -.->|"chunk+embed"| Embed
  Embed -.->|"upsert_vectors"| VecIdx

  RAGQuery -.->|"search"| Retrieve
  Retrieve -.->|"reads"| VecIdx
  Retrieve -.->|"top_k_context"| PromptBuild
  RAGQuery -.->|"question"| PromptBuild
  PromptBuild -.->|"augmented_prompt"| LLMReuse
  Disk -.->|"loads_*.gguf"| LLMReuse
  LLMReuse -.->|"answer+cites"| RAGResp
end

%% Link RAG notebooks to subsystem
Rag1 -.->|"explores"| Retrieve
Rag2 -.->|"explores"| PromptBuild
Rag3 -.->|"explores"| VecIdx
WebScrap -.->|"implements"| Scrape

%% =======================
%% Click Events (from component_mapping)
%% =======================
click Readme "https://github.com/codex0870/nirf-college-ranking/blob/main/README.md"
click QuantNB "https://github.com/codex0870/nirf-college-ranking/blob/main/gemma_quantization.ipynb"
click QuantNBv2 "https://github.com/codex0870/nirf-college-ranking/blob/main/gemma_quantizationv2_0.ipynb"
click MainPy "https://github.com/codex0870/nirf-college-ranking/blob/main/main.py"
click PyApp "https://github.com/codex0870/nirf-college-ranking/blob/main/main.py"
click UseSh "https://github.com/codex0870/nirf-college-ranking/blob/main/use_gguf_model.sh"
click ShHelper "https://github.com/codex0870/nirf-college-ranking/blob/main/use_gguf_model.sh"
click Reqs "https://github.com/codex0870/nirf-college-ranking/blob/main/requirements.txt"
click Rag1 "https://github.com/codex0870/nirf-college-ranking/blob/main/rag_1.ipynb"
click Rag2 "https://github.com/codex0870/nirf-college-ranking/blob/main/rag_2.ipynb"
click Rag3 "https://github.com/codex0870/nirf-college-ranking/blob/main/rag_3.ipynb"
click WebScrap "https://github.com/codex0870/nirf-college-ranking/blob/main/web_scrap.ipynb"
click Scrape "https://github.com/codex0870/nirf-college-ranking/blob/main/web_scrap.ipynb"

%% =======================
%% Styles
%% =======================
classDef actor fill:#ffffff,stroke:#111827,stroke-width:1px,color:#111827
classDef external fill:#f3f4f6,stroke:#6b7280,stroke-width:1px,color:#111827
classDef runtime fill:#dbeafe,stroke:#2563eb,stroke-width:1px,color:#0f172a
classDef offline fill:#dcfce7,stroke:#16a34a,stroke-width:1px,color:#052e16
classDef rag fill:#ffedd5,stroke:#f97316,stroke-width:1px,color:#7c2d12
classDef store fill:#e5e7eb,stroke:#374151,stroke-width:1px,color:#111827
classDef doc fill:#ede9fe,stroke:#7c3aed,stroke-width:1px,color:#2e1065
classDef compute fill:#cffafe,stroke:#0891b2,stroke-width:1px,color:#083344
```
