How davebcn87/pi-autoresearch works
What this project is (type + purpose)
This repo is a pi agent add-on consisting of:
• A pi extension (extensions/pi-autoresearch) that provides domain-agnostic experiment-loop infrastructure:
• tools: init_experiment, run_experiment, log_experiment
• UI: a status widget and a /autoresearch dashboard
• persistent storage: appends run results to autoresearch.jsonl in the user’s target project
• A pi skill (skills/autoresearch-create) that provides domain-specific orchestration:
• asks/infer experiment goal, command, metric, scope
• writes session artifacts: autoresearch.md and autoresearch.sh
• starts the autonomous loop
What the file tree implies architecturally
From the tree:
• extensions/pi-autoresearch/index.ts
• single extension entrypoint (likely registers tools + UI routes/panels + reads/writes session files)
• skills/autoresearch-create/SKILL.md
• skill definition/instructions (pi skill metadata + prompt-like behavior)
• Root package.json / package-lock.json
• Node/TypeScript packaging for the extension
Key architectural inference: two-layer separation
• Extension = platform capabilities (tooling, persistence, UI)
• Skill = policy + domain knowledge (what to optimize, how to benchmark, which files matter)
What the README adds (runtime components + data flow)
Runtime loop described:
Skill writes
• autoresearch.md (human/agent resumable “session doc”)
• autoresearch.sh (benchmark runner that emits METRIC name=number)
Repeated loop
• edit → commit → run_experiment → log_experiment → keep/revert → repeat
Extension persists
• autoresearch.jsonl append-only history (per branch)
UI surfaces
• widget (always visible)
• /autoresearch dashboard (results table, best run)
External dependencies that matter in architecture:
• Shell execution environment (runs arbitrary commands)
• Git (auto-commit, branch-aware sessions, revert/keep decisions)
• Target project repo (where files live and are mutated)
Pi Agent Runtime
• hosts skills + extensions, provides tool invocation and UI surface
Skill: autoresearch-create
• config gathering / inference
• session bootstrap (branch creation, file creation)
• loop controller (decision logic: keep vs revert)
Extension: pi-autoresearch (TypeScript)
• Tool handlers:
• init_experiment
• run_experiment
• log_experiment
• UI:
• status widget
• /autoresearch dashboard
• Session I/O:
• read/write autoresearch.md
• append autoresearch.jsonl
User Project Workspace (the optimized repository)
• source files under test
• generated artifacts:
• autoresearch.md
• autoresearch.sh
• autoresearch.jsonl
Git (VCS)
• branches per session
• auto-commit after changes
• revert/checkout when discarding runs
Shell / Command Runner
• executes the benchmark command(s) (e.g., pnpm test, lighthouse …)
• returns stdout/stderr + exit code + wall-clock duration
• Metric parser (extracts METRIC name=number from output)
• Result evaluator (compares best/kept vs discarded given direction ↑/↓)
Bootstrap phase
• Pi Agent → Skill (/skill:autoresearch-create)
• Skill → User Project Workspace (writes autoresearch.md, autoresearch.sh)
• Skill → Git (create branch, initial commit)
• Skill → Extension tool: init_experiment (store config / initialize session)
Loop phase (repeat)
• Skill → User Project Workspace (edits files / applies changes)
• Skill → Git (commit changes)
• Skill → Extension tool: run_experiment
• Extension run_experiment → Shell/Command Runner (exec command)
• Shell → Extension (stdout/stderr, duration, exit code)
• Extension → (Metric parser) → numeric metric
• Skill (or extension) decides keep/revert (make this explicit)
• Skill → Git (revert/checkout if discard, or keep commit if good)
• Skill → Extension tool: log_experiment
• Extension log_experiment → User Project Workspace (append autoresearch.jsonl)
• Extension UI (widget/dashboard) ← reads from autoresearch.jsonl (and maybe autoresearch.md)
Monitoring
• Developer (human) → Pi UI (/autoresearch, widget)
• UI → Extension → reads autoresearch.jsonl → renders results/best run
Key architectural patterns
• Separation of concerns
• *Skill* = policy/orchestration (domain knowledge)
• *Extension* = infrastructure (tools, persistence, UI)
• Event log / append-only history
• autoresearch.jsonl as the durable run ledger
• Resumability
• autoresearch.md + autoresearch.jsonl allow continuation after restarts/context loss
• Branch isolation
Relevant technologies
• Extension: TypeScript/Node (index.ts, package.json)
• Skill: pi Skill definition (SKILL.md)
• Persistence: JSONL log + Markdown session doc
• Execution: shell commands + parsing METRIC name=number
• SCM: Git (commit, branch, revert)
Labels
• init_experiment, run_experiment, log_experiment
• autoresearch.md, autoresearch.sh, autoresearch.jsonl
• Payloads:
• “stdout/stderr + duration”
• “parsed metric (name, value, unit, direction)”
• “commit hash, status, description”
Don’t under-model: recommended component breakdown (maximally clear)
• Tool: run_experiment (exec + timing + output capture + metric parsing)
• Tool: log_experiment (append JSONL + auto-commit + notify UI)
• UI: widget
• UI: dashboard route /autoresearch
• Storage: jsonl reader/writer
High-level (static)
• Pi Agent Runtime
• Skill: autoresearch-create
• Extension: pi-autoresearch
• tools: init/run/log
• UI: widget + dashboard
• User Project Workspace
• autoresearch.md, autoresearch.sh, autoresearch.jsonl
• Git
• Shell/Command Runner
Sequence (one iteration)
Human → Skill → (edit files) → Git commit → run_experiment → Shell exec → metric parse → keep/revert via Git → log_experiment → append JSONL → UI refresh.