How karakuri-ai/wandb-fully-connected-2025-karakuri-workshop works
Project type and purpose
• This is a Python-based demo/workshop tool—a “Customer Support AI” application built for a W&B (Weights & Biases) workshop.
• It ingests FAQ data, builds embeddings, serves a chat interface to answer user questions via OpenAI’s API, and logs experiments/metrics to W&B.
File structure
• Top-level directories/files:
• data/ (CSV sources: faq.csv, faq-embedding.csv, eval-onboarding.csv, feedback-logs.csv)
• notebooks/ (workshop.ipynb for interactive demos)
• src/ (core Python modules: chat.py, client.py, config.py, prompts.py, prompt_improvement.py, run_evaluation.py, run_improvement.py, tools.py)
• .streamlit/ (streamlit config, if a UI is used)
• pyproject.toml & uv.lock (dependency management with “uv”)
• Makefile, env.example, README.md
• Patterns: single codebase, modularized by functionality (chat, client adapters, prompt logic, data-tools, evaluation/improvement scripts).
README for additional insights
• Environment variables: OPENAI_API_KEY, WANDB_API_KEY, WEAVE_PROJECT_NAME
• Toolchain: Python ≥3.11, “uv sync” for dependency install
• Usage: runs scripts to evaluate prompts and improve them, uses Google Colab for live demo.
a. Data Source layer
• CSV Files in data/ → loaded by tools.py → embeddings stored (locally or in-memory)
b. Embedding & Vector Store
• prompt_improvement.py & tools.py call OpenAI Embedding API → produce faq-embedding.csv
c. Chat Interface & Prompt Assembly
• chat.py uses client.py (OpenAI client) + prompts.py (templates + retrieval) → builds completion requests
d. OpenAI API
• External service: ChatCompletion & Embedding endpoints
e. Experiment Logging
• config.py sets up W&B → run_evaluation.py & run_improvement.py log to W&B project
f. Notebook Layer
• notebooks/workshop.ipynb demonstrates end-to-end usage interactively
Architectural patterns & design principles
• Modular layering:
• “tools” for data ingestion & embedding
• “prompts” for template management
• “client” for external API abstraction
• “chat” for orchestrating input → retrieval → LLM call → postprocessing
• “run_*” scripts for batch experiments & improvements
• Separation of concerns: data, model, evaluation, logging
• Extensibility hooks: swap out embedding, add new prompt modules, integrate new evaluation metrics