# karthik-codex/autogen_graphrag_ollama architecture

> Architecture diagram of the GitHub repository karthik-codex/autogen_graphrag_ollama, generated by GitDiagram from its file tree, README and a sample of source files (last updated 2025-08-18). It is an AI-made overview: check details against the source.

- Interactive diagram: https://gitdiagram.com/karthik-codex/autogen_graphrag_ollama
- Repository: https://github.com/karthik-codex/autogen_graphrag_ollama

## Overview

##### Project type and purpose
- This is a local Retrieval-Augmented-Generation (RAG) “Superbot” that glues together:
- GraphRAG (knowledge‐graph–backed search & retrieval)
- AutoGen agents (multi-agent orchestration via function calling)
- Ollama local LLMs (for embedding & inference)
- Chainlit (interactive web UI for continuous conversation)
- It’s a full-stack, single-repository Python application that runs entirely offline.

##### File structure
- Top level:
- appUI.py – Chainlit UI entrypoint (Web server + conversation loop)
- chainlit.md – Chainlit usage notes
- requirements.txt – Python dependencies (AutoGen, graphrag, chainlit, etc.)
- utils/ – Overrides, helpers, and ingestion scripts
– chainlit_agents.py – definitions of AutoGen agent workflows & function-call handlers
– embedding.py & openai_embeddings_llm.py – monkey-patched modules to route GraphRAG embeddings & LLM calls to Ollama
– pdf_to_markdown.py – ingests PDF files into Markdown for indexing
– settings.yaml – configuration for GraphRAG indexing (paths, embedding model names)
- Deployment/config files: none beyond settings.yaml; the project relies on external “ollama serve” and “litellm” proxy.

##### Key insights from README
- Workflow: ingest docs → build GraphRAG index → launch Lite-LLM proxy → start Chainlit UI → user interacts → AutoGen agents orchestrate RAG calls → responses returned.
- Emphasis on non-OpenAI function calling via a Lite-LLM proxy that fronts Ollama models.

##### Main components
a. User & UI Layer
- User’s browser or Chainlit client → HTTP/WebSocket → appUI.py (Chainlit server)
b. Orchestration Layer (AutoGen Agents)
- chainlit_agents.py defines multiple agents (retriever, reasoner, action planner)
- Function-call interface to GraphRAG and embedding services
c. Retrieval & Embedding Layer (GraphRAG)
- Knowledge graph index and vector store (on local filesystem)
- pdf_to_markdown → document ingestion pipeline
- embedding.py & openai_embeddings_llm.py override GraphRAG’s calls to use local Ollama embedding models
d. LLM Inference Layer (Ollama + Lite-LLM proxy)
- Ollama serve (models: mistral, llama3)
- Lite-LLM proxy (litellm) exposing an OpenAI-compatible HTTP interface
e. Data Layer
- Local folder structure: ./input for source docs, GraphRAG index files, logs
- settings.yaml for configuration (model names, index paths)

##### Relationships and data flow
1. User sends query → Chainlit UI (appUI.py)
2. UI dispatches to AutoGen agent orchestration (chainlit_agents.py)
3. Agent: embed query → call Lite-LLM / Ollama embeddings via proxy
4. GraphRAG uses embeddings to search vector store / knowledge graph → returns relevant context
5. Agent composes prompt + context → sends to LLM via Lite-LLM proxy for generation
6. Agent processes response, optionally calls functions (e.g., graph updates) → final answer
7. Chainlit UI returns answer to user, displays in real time

##### Architectural patterns & design principles
- Modular multi-agent architecture (AutoGen) separated from UI
- Retrieval-augmented generation (GraphRAG + vector search)
- Adapter/Override pattern: utils/embedding.py injects custom Ollama embedding logic into GraphRAG codebase
- Function-call pattern for robust agent-to-service interactions
- Clear separation of concerns:
- UI
- Orchestration
- Retrieval/Indexing
- Embedding/In-ference

## Diagram (Mermaid)

```mermaid
graph TB
    U[("User")]:::user

    subgraph "UI Layer"
        UI["Chainlit UI<br/>(appUI.py)"]:::code
    end

    subgraph "Orchestration Layer"
        AG["AutoGen Agent Orchestrator<br/>(chainlit_agents.py)"]:::code
    end

    subgraph "Retrieval/Indexing Layer"
        ING["Document Ingestion Pipeline<br/>(pdf_to_markdown.py)"]:::code
        CFG["GraphRAG Configuration<br/>(settings.yaml)"]:::code
        OV1["Embedding Override<br/>(utils/embedding.py)"]:::code
        OV2["LLM Embedding Adapter<br/>(openai_embeddings_llm.py)"]:::code
        INDEX["Knowledge Graph & Vector Store"]:::data
    end

    subgraph "Inference/Embedding Layer"
        PROXY["Lite-LLM Proxy<br/>(litellm)"]:::service
        OLLAMA["Ollama Serve<br/>(Mistral, Llama3)"]:::service
    end

    subgraph "Data Layer"
        DOCS["./input (Source Docs)"]:::data
        LOGS["Logs & Index Files"]:::data
    end

    subgraph "Documentation & Metadata"
        RE["README.md"]:::doc
        REQ["requirements.txt"]:::doc
        MD["chainlit.md"]:::doc
        IMG["images/"]:::doc
    end

    U -->|"HTTP/WebSocket"| UI
    UI -->|"dispatch query"| AG
    AG -->|"embed(query)"| OV1
    OV1 -->|"delegate to"| OV2
    OV2 -->|"embed via"| PROXY
    PROXY -->|"forward to"| OLLAMA
    AG -->|"search(context)"| INDEX
    ING -->|"ingest docs into"| INDEX
    CFG -->|"configures"| ING
    AG -->|"generate(answer)"| PROXY
    AG -->|"function calls"| INDEX
    AG -->|"return response"| UI

    classDef code fill:#cfe2f3,stroke:#6c8ebf
    classDef data fill:#d9ead3,stroke:#6aa84f
    classDef service fill:#d0e0e3,stroke:#38761d
    classDef user fill:#fce5cd,stroke:#e69138
    classDef doc fill:#eeeeee,stroke:#aaaaaa

    click UI "https://github.com/karthik-codex/autogen_graphrag_ollama/blob/main/appUI.py"
    click AG "https://github.com/karthik-codex/autogen_graphrag_ollama/blob/main/utils/chainlit_agents.py"
    click ING "https://github.com/karthik-codex/autogen_graphrag_ollama/blob/main/utils/pdf_to_markdown.py"
    click CFG "https://github.com/karthik-codex/autogen_graphrag_ollama/blob/main/utils/settings.yaml"
    click OV1 "https://github.com/karthik-codex/autogen_graphrag_ollama/blob/main/utils/embedding.py"
    click OV2 "https://github.com/karthik-codex/autogen_graphrag_ollama/blob/main/utils/openai_embeddings_llm.py"
    click REQ "https://github.com/karthik-codex/autogen_graphrag_ollama/blob/main/requirements.txt"
    click MD "https://github.com/karthik-codex/autogen_graphrag_ollama/blob/main/chainlit.md"
    click RE "https://github.com/karthik-codex/autogen_graphrag_ollama/blob/main/README.md"
    click IMG "https://github.com/karthik-codex/autogen_graphrag_ollama/tree/main/images/"
```
