How alibaba/skill-up works
skill-up is a CLI for evaluating agent skills, agents, and workspaces. The main workflow loads and validates an evaluation, prepares a runtime and case environment, runs an agent, grades its response and workspace changes, then writes structured reports. Configuration and engine adapters support multiple built-in and custom agents; optional MCP provisioning and event logging extend runs. The bundled skill-upper plugin supports an evidence-driven improvement loop. Some branches are mapped from README claims or import references because their implementation excerpts were not provided.
Main entry points: cli CLI commands · evaluator.go Case evaluation · index.js skill-upper plugin · runner.go Run orchestration
Components 19
CLI and configuration
- CLI commands
internal/cli - Eval configuration
internal/config - Custom engine config
internal/customengine
Evaluation execution
- Run orchestration
internal/runner/runner.go - Case evaluation
internal/evaluator/evaluator.go - Agent engines
internal/agent - Transcript model
pkg/transcript/transcript.go
Runtime and integrations
- Runtime providers
internal/runtime - MCP provisioning
internal/mcp - Skill installation
internal/skill/installer.go
Judging and reporting
- Judge strategies
internal/judge - Evaluation reports
internal/report - Run event log
internal/evalevent
Skill evolution
- skill-upper plugin
plugins/dsh-skill-up/index.js - Plugin run helpers
plugins/dsh-skill-up/lib/core.js - Evidence comparison
plugins/dsh-skill-up/lib/observer.js
Other components
- CLI user
- OpenSandbox service
- Docker runtime
Connections 18
- CLI user → CLI commands: runs commands
- CLI commands → Eval configuration: loads config
internal/cli/run.go - CLI commands → Agent engines: selects engine
internal/cli/run.go - CLI commands → Run orchestration: starts evaluation
internal/cli/run.go - CLI commands → Run event log: streams events
internal/cli/run_events.go - Eval configuration → Custom engine config: configures custom engine
internal/config/customengine.go - Run orchestration → Case evaluation: evaluates plan
internal/runner/runner.go - Run orchestration → Evaluation reports: writes results
internal/runner/runner.go - Case evaluation → Agent engines: runs agent
internal/evaluator/evaluator.go - Case evaluation → Runtime providers: prepares runtime
internal/evaluator/evaluator.go - Case evaluation → MCP provisioning: provisions MCP
internal/evaluator/evaluator.go - Case evaluation → Judge strategies: grades outcome
internal/evaluator/evaluator.go - Agent engines → Transcript model: records conversation
internal/agent/codex.go - Runtime providers → OpenSandbox service: uses service
internal/runtime/opensandbox.go - Runtime providers → Docker runtime: uses container
internal/runtime/docker.go - skill-upper plugin → Plugin run helpers: uses helpers
plugins/dsh-skill-up/index.js - skill-upper plugin → Evidence comparison: compares results
plugins/dsh-skill-up/index.js - skill-upper plugin → CLI commands: drives eval loop
README.md