How remote-controlled-man/skillfit works
skillfit is a CLI for checking agent-configuration health, measuring configuration effects on task benches, and installing only selected profiles. The main evaluation path is CLI → eval command → harness runs paired baseline/treatment trials, grades outputs, and summarizes evidence; trigger mode additionally captures whether an installed skill was invoked. Other product branches cover bench authoring/validation, health checks, usage receipts, and managed installation. The tree and excerpts support the major modules, but some command and harness wiring is unsampled, so uncertain internal links are omitted.
Main entry points: runner.ts Trial runner · cli.ts Command router · eval.ts Eval command · executors Agent executors
Components 19
CLI entry
- Command router
src/cli.ts
Evaluation
- Eval command
src/commands/eval.ts - Trial runner
src/harness/runner.ts - Prompt builder
src/harness/prompt.ts - Agent executors
src/harness/executors - Verifier summary
src/harness/verifier-summary.ts - Blind judge
src/harness/judge.ts - Trial statistics
src/harness/stats.ts - Run reports
src/harness/report.ts
Configuration
- Setup health
src/commands/doctor.ts - Usage receipts
src/commands/report.ts - Profile installer
src/commands/install.ts - Agent definitions
src/agents.ts
Bench authoring
- Bench commands
src/commands - Bench tasks
benches
Other components
- Operator
- Agent CLI
- Model API
- Local agent setup
Connections 22
- Operator → Command router: runs command
- Command router → Eval command: dispatches eval
- Command router → Setup health: dispatches doctor
- Command router → Usage receipts: dispatches report
- Command router → Profile installer: dispatches install
- Command router → Bench commands: dispatches bench
- Eval command → Bench tasks: selects tasks
- Eval command → Trial runner: runs trials
- Trial runner → Prompt builder: builds prompts
- Trial runner → Agent executors: executes trial
- Agent executors → Agent CLI: invokes agent
- Agent executors → Model API: calls model
- Trial runner → Verifier summary: summarizes verdict
- Trial runner → Blind judge: judges pairs
- Trial runner → Trial statistics: summarizes trials
- Trial runner → Run reports: builds report
- Bench commands → Bench tasks: creates or checks
- Setup health → Agent definitions: loads definitions
- Setup health → Local agent setup: inspects setup
- Usage receipts → Local agent setup: reads history
- Profile installer → Agent definitions: selects targets
- Profile installer → Local agent setup: writes configuration