How olehxch/llm-code-examples works
Project Type and Purpose
• This is an educational “code-examples” repository, not a running product. It demonstrates patterns for working with large language models (LLMs).
• It comprises three logical sub-projects:
• LLM-Book Examples: OpenAI API + ChromaDB vector store.
• AutoGEN Examples: agent orchestration with Ollama + code executors (local/Docker).
• Nvidia Examples: integration with NVIDIA NIM inference microservices.
File Structure Analysis
• Top-level directories:
• autogen-examples/
• llm-book-examples/
• nvidia-examples/
• Supporting files:
• pyproject.toml, uv.lock → Python project metadata + uv package-manager lock.
• .python-version → Python runtime version.
• .gitignore, README.md → documentation + ignore rules.
• Each example directory contains Python scripts, a requirements file (req.txt) and expects a .env with API keys.
• Patterns:
• Each sub-project is self-contained, follows a consistent naming convention for numbered examples.
• The AutoGEN examples introduce sub-components: agents, human-input handlers, code executors.
Key Architectural Insights from README
• LLM-Book: uses OpenAI Chat or Completion API, optional embeddings & vector DB calls.
• AutoGEN: uses Microsoft’s Autogen framework + Ollama as local LLM host; supports two agent types (AssistantAgent, UserProxyAgent), multiple execution modes (local vs Docker, CLI vs Jupyter).
• Nvidia: calls NVIDIA NIM inference microservices via OpenAI-compatible API endpoints.
Components
a. Local Environment Layer
– Python 3.12 runtime
– uv package manager + installed dependencies
– .env files for API keys
– LLM-Book Examples
– AutoGEN Examples
– Nvidia Examples
c. External Services Layer
– OpenAI API (chat, completion, embeddings)
– ChromaDB vector database
– Ollama local LLM host
– Docker engine (for containerized code execution)
– Jupyter kernel (for Jupyter executor)
– NVIDIA NIM inference microservices
d. Data Stores
– embeddings.json / data.txt in some examples
– vector DB (ChromaDB)
– function‐library vector DB
Relationships and Data Flows
• Python script ► loads dependencies ► reads .env ► builds prompt or agent configuration.
• LLM-Book examples:
• call OpenAI API ► receive text or embeddings ► optionally write/read ChromaDB.
• AutoGEN examples:
• instantiate agents ► orchestrate conversation patterns ► for code messages call code executor (local or Docker or Jupyter) ► return output to agent conversation.
• Nvidia examples:
• call NVIDIA NIM API ► receive response (streaming or full) ► process/display.
Architectural Patterns and Design Principles
• Modularity: examples grouped by capabilities.
• Layered external integrations: clear separation of API calls, vector DB, code execution.
• Agent-based orchestration (AutoGEN): ConversableAgent pattern, strategy for human input and termination.
• Vector retrieval pattern: store embeddings, query DB by cosine similarity.
• Function calling optimization: selective retrieval of functions from vector DB.