How karthik-codex/autogen_graphrag_ollama works
Project type and purpose
• This is a local Retrieval-Augmented-Generation (RAG) “Superbot” that glues together:
• GraphRAG (knowledge‐graph–backed search & retrieval)
• AutoGen agents (multi-agent orchestration via function calling)
• Ollama local LLMs (for embedding & inference)
• Chainlit (interactive web UI for continuous conversation)
• It’s a full-stack, single-repository Python application that runs entirely offline.
File structure
• Top level:
• appUI.py – Chainlit UI entrypoint (Web server + conversation loop)
• chainlit.md – Chainlit usage notes
• requirements.txt – Python dependencies (AutoGen, graphrag, chainlit, etc.)
• utils/ – Overrides, helpers, and ingestion scripts
– chainlit_agents.py – definitions of AutoGen agent workflows & function-call handlers
– embedding.py & openai_embeddings_llm.py – monkey-patched modules to route GraphRAG embeddings & LLM calls to Ollama
– pdf_to_markdown.py – ingests PDF files into Markdown for indexing
– settings.yaml – configuration for GraphRAG indexing (paths, embedding model names)
• Deployment/config files: none beyond settings.yaml; the project relies on external “ollama serve” and “litellm” proxy.
Key insights from README
• Workflow: ingest docs → build GraphRAG index → launch Lite-LLM proxy → start Chainlit UI → user interacts → AutoGen agents orchestrate RAG calls → responses returned.
• Emphasis on non-OpenAI function calling via a Lite-LLM proxy that fronts Ollama models.
Main components
a. User & UI Layer
• User’s browser or Chainlit client → HTTP/WebSocket → appUI.py (Chainlit server)
b. Orchestration Layer (AutoGen Agents)
• chainlit_agents.py defines multiple agents (retriever, reasoner, action planner)
• Function-call interface to GraphRAG and embedding services
c. Retrieval & Embedding Layer (GraphRAG)
• Knowledge graph index and vector store (on local filesystem)
• pdf_to_markdown → document ingestion pipeline
• embedding.py & openai_embeddings_llm.py override GraphRAG’s calls to use local Ollama embedding models
d. LLM Inference Layer (Ollama + Lite-LLM proxy)
• Ollama serve (models: mistral, llama3)
• Lite-LLM proxy (litellm) exposing an OpenAI-compatible HTTP interface
e. Data Layer
• Local folder structure: ./input for source docs, GraphRAG index files, logs
• settings.yaml for configuration (model names, index paths)
Relationships and data flow
1. User sends query → Chainlit UI (appUI.py)
2. UI dispatches to AutoGen agent orchestration (chainlit_agents.py)
3. Agent: embed query → call Lite-LLM / Ollama embeddings via proxy
4. GraphRAG uses embeddings to search vector store / knowledge graph → returns relevant context
5. Agent composes prompt + context → sends to LLM via Lite-LLM proxy for generation
6. Agent processes response, optionally calls functions (e.g., graph updates) → final answer
7. Chainlit UI returns answer to user, displays in real time
Architectural patterns & design principles
• Modular multi-agent architecture (AutoGen) separated from UI
• Retrieval-augmented generation (GraphRAG + vector search)
• Adapter/Override pattern: utils/embedding.py injects custom Ollama embedding logic into GraphRAG codebase
• Function-call pattern for robust agent-to-service interactions
• Clear separation of concerns:
• UI
• Orchestration
• Retrieval/Indexing
• Embedding/In-ference