How nerdprog/prompt_lab works
PromptLab is a web application for iteratively optimizing prompts: users submit a task, confirm extracted intent, and receive ranked prompts with a PDF report. Its central service coordinates rubric creation, candidate generation, bandit selection, Performer runs, Judge scoring, feedback-driven refinement, and final evaluation while tracking session state. The README documents additional capabilities such as intent-fidelity scoring, provider health checks, cancellation, and progress reporting. Relationships below are based on the README and SOURCE INDEX references; unsampled implementation details are not inferred beyond those references.
Main entry points: optimization_service.py Optimization service · routes_optimization.py Optimization API · config.py Runtime configuration · app.js Browser UI
Components 22
Web interface
- Browser UI
app/static/js/app.js - Web page
app/templates/index.html - Optimization API
app/api/routes_optimization.py
Optimization workflow
- Task schemas
app/schemas/task.py - Input validation
app/utils/validation.py - Optimization service
app/services/optimizer/optimization_service.py - Task understanding
app/services/intent/task_understanding.py - Rubric generation
app/services/rubric/rubric_generator.py - Candidate generation
app/services/optimizer/candidate_generator.py - Feedback optimization
app/services/optimizer/feedback_optimizer.py - Session insights
app/services/insights/insight_store.py
Prompt evaluation
- Intent fidelity
app/services/intent/intent_fidelity.py - UCB selector
app/services/selector/ucb.py
Model providers
- Performer model
app/services/llm/performer.py - Judge model
app/services/llm/judge.py - Optimizer model
app/services/llm/optimizer.py - Provider health
app/services/providers/health.py - Runtime configuration
app/core/config.py
Session and reporting
- In-memory sessions
app/state/session_state.py - PDF report
app/services/pdf/report_generator.py
Other components
- User
- External LLM providers
Connections 30
- User → Browser UI: submits task
README.md - Browser UI → Web page: uses page
- Browser UI → Optimization API: calls API
README.md - Optimization API → Optimization service: dispatches optimization
app/api/routes_optimization.py - Optimization API → In-memory sessions: reads session
app/api/routes_optimization.py - Optimization API → Provider health: checks providers
app/api/routes_optimization.py - Optimization API → PDF report: serves report
app/api/routes_optimization.py - Optimization service → Input validation: validates prompt
app/services/optimizer/optimization_service.py - Optimization service → Task understanding: extracts intent
app/services/optimizer/optimization_service.py - Task understanding → Task schemas: creates intent
app/services/intent/task_understanding.py - Optimization service → Rubric generation: builds rubric
app/services/optimizer/optimization_service.py - Optimization service → Candidate generation: generates candidates
app/services/optimizer/optimization_service.py - Candidate generation → Optimizer model: requests prompts
app/services/optimizer/candidate_generator.py - Candidate generation → Intent fidelity: checks fidelity
app/services/optimizer/candidate_generator.py - Optimization service → UCB selector: selects candidates
app/services/optimizer/optimization_service.py - Optimization service → Performer model: executes prompts
app/services/optimizer/optimization_service.py - Optimization service → Judge model: evaluates responses
app/services/optimizer/optimization_service.py - Optimization service → Feedback optimization: refines candidates
app/services/optimizer/optimization_service.py - Optimization service → Session insights: updates insights
app/services/optimizer/optimization_service.py - Feedback optimization → Session insights: uses evidence
app/services/optimizer/feedback_optimizer.py - Optimization service → In-memory sessions: updates session
app/services/optimizer/optimization_service.py - Optimization service → PDF report: creates report
app/services/optimizer/optimization_service.py - Performer model → External LLM providers: calls Gemini
README.md - Optimizer model → External LLM providers: calls Gemini
README.md - Judge model → External LLM providers: calls Groq
README.md - Provider health → External LLM providers: probes providers
app/services/providers/health.py - Performer model → Runtime configuration: reads settings
app/services/llm/performer.py - Optimizer model → Runtime configuration: reads settings
app/services/llm/optimizer.py - Judge model → Runtime configuration: reads settings
app/services/llm/judge.py - Provider health → Runtime configuration: reads settings
app/services/providers/health.py