How rishit-singh/tinytune works
Project Type and Purpose
• TinyTune is an open‑source Python library for prompt engineering and workflow orchestration over Large Language Models (LLMs).
• It exposes modular “prompt jobs,” “pipelines,” and “tools” so users can build, reuse, and chain LLM‐driven tasks in a clean, model‑agnostic way.
File Structure Analysis
• Top‑level:
• README.md, pyproject.toml, LICENSE – project metadata & packaging.
• docs/ – full documentation site (core concepts, API reference, advanced usage).
• examples/ – sample scripts showing real‑world usage.
• src/tinytune/ – core library code:
– contexts/ (LLMContext implementations)
– llmcontext.py (abstract base)
– pipeline.py (Pipeline orchestration)
– prompt.py & util/prompt.py (Message, prompt_job decorator)
– tool.py (tool decorator and execution hooks)
• Config files: pyrightconfig.json (type checking), _gitignore.
Key Components
a. User Application Layer
• User code (examples/** or consumer app)
• Defines:
• An LLMContext instance (e.g., GPTContext)
• @tool‑decorated Python functions
• @prompt_job‑decorated prompt jobs
• A Pipeline that chains prompt jobs
b. TinyTune Core Library
• Contexts Module: LLMContext base + concrete subclasses (GPTContext, OllamaContext, etc.)
• Prompt Module: decorator @prompt_job, Message data model
• Pipeline Module: orchestrates job sequence, passes prevResult
• Tool Module: decorator @tool, metadata registry, invocation mechanism
• Util Module: shared helpers (e.g., prompt templating)
c. External Services
• Third‑party LLM provider APIs (OpenAI, Groq, Replicate, etc.)
• Any external systems invoked by user‑defined tools (databases, web APIs)
Relationships & Data Flow
1. User App → Instantiates an LLMContext.
2. User App → Decorates functions as tools and prompt jobs.
3. Pipeline (user code) invokes Pipeline.Run():
a. For each prompt job:
i. Pipeline calls job(context, prevResult).
ii. Prompt job uses context.Prompt(Message).Run() → HTTP request to LLM API.
iii. LLM API returns response → context captures as Message object.
iv. Pipeline sets prevResult ← response → next job.
b. If a prompt job invokes a tool:
i. TinyTune’s tool registry intercepts call.
ii. Runs user function locally or posts to external service.
iii. Returns tool output into prompt/job.
4. Final output returned to user app.
Architectural Patterns & Principles
• Plugin / Decorator Pattern: @prompt_job and @tool annotations register user code.
• Pipeline / Chain of Responsibility: sequential job execution with shared context.
• Adapter Pattern for LLMContext: abstracts over multiple LLM backends.
• Separation of Concerns: user code vs. core orchestration vs. external API calls.