How bespokelabsai/curator works
Main Components
• User/Application Interface:
– This is where users (or developers) interact with the library via the exposed APIs (such as the curator.LLM class), CLI commands, or even through example scripts.
• LLM Integration Module:
– Represented by components in src/bespokelabs/curator/llm and related prompt formatter and response parser logic.
– Shows the flow of prompts to external LLM services.
• Request Processing:
– Located in src/bespokelabs/curator/request_processor. This module handles both online and offline (batch) request processing, with sub-components for different providers (Anthropic, OpenAI, LiteLLM, etc.).
– Include the configuration elements (like rate limits and the _default_rate_limits.json file) and factory pattern logic.
• Code Execution Engine:
– Found under src/bespokelabs/curator/code_executor. This component abstracts code execution into several pluggable backends (multiprocessing, Docker, Ray, and e2b).
• Blocks and Specialized Data Generation:
– Represent the “blocks” (e.g., RAFT in src/bespokelabs/curator/blocks), which encapsulate domain-specific operations like fine-tuning or retrieval augmented fine-tuning (RAFT).
• Status Tracking and Caching:
– Modules such as src/bespokelabs/curator/status_tracker and database modules (db.py & code_executor/db.py) are critical for tracking ongoing processes and caching responses (including retry logic).
• Viewer/Telemetry:
– Illustrate the Curator Viewer (supported by the server.js file under viewer) that streams data and supports real-time monitoring.
– Additionally, show the telemetry client (under telemetry) that sends anonymized usage metrics.
Relationships and Data Flows
– Show how the Request Processor dispatches requests to external APIs (e.g., OpenAI, Anthropic, Gemini) via connectors.
– Make sure to show the integration between the core engine and the Viewer, where data (or logs) are sent for real-time visualization.
– Also capture secondary flows such as caching and status tracking that monitor request lifecycles and outcomes.
Architectural Patterns & Design Principles
– Modular Design: Each main functionality (LLM calls, request processing, code execution) exists as separate modules/packages.
– Pluggable Backends: Use of factory patterns and strategy patterns especially in the code_executor package, allowing different backend implementations.
– Asynchronous and Batch Processing: Stress how the system supports large-scale synthetic data generation with batch mode support.
– Extensibility: The project has clearly defined integration points for adding new LLM providers, new code backends, and even custom prompts/data parsers.
Incorporate External Dependencies and Technologies
– Label external APIs/integrations for OpenAI, Anthropic, Gemini, kluster.ai, and potentially vLLM.
– Emphasize any supporting tools or libraries (for example, use of pydantic for structured outputs, HuggingFace’s Dataset API for data manipulation, and the CLI for viewer interactions).