How cmu-db/ottertune works
OtterTune’s workflow is to collect DBMS configuration and performance observations, upload them to the service, normalize them into stored tuning results, and use historical workload data to recommend better knob settings. The graph preserves the documented cross-session learning capability and the principal analysis families referenced by the task modules. Relationships involving unsampled modules are limited to references in the SOURCE INDEX; optional client-to-server transport is documented only at the controller’s upload entry point.
Main entry points: async_tasks.py Recommendation tasks · Main.java Controller runner · periodic_tasks.py Workload processing · models.py Tuning records
Components 22
DBMS controller
- Controller runner
client/controller/src/main/java/com/controller/Main.java - Controller config
client/controller/src/main/java/com/controller/ControllerConfiguration.java - DB collectors
client/controller/src/main/java/com/controller/collectors/DBCollector.java - Result uploader
client/controller/src/main/java/com/controller/ResultUploader.java - JSON utilities
client/controller/src/main/java/com/controller/util/JSONUtil.java
Tuning service
- Service views
server/website/website/views.py - Tuning records
server/website/website/models.py - DBMS parser dispatch
server/website/website/db/parser.py - Base DBMS parser
server/website/website/db/base/parser.py - Vendor parsers
server/website/website/db - Recommendation tasks
server/website/website/tasks/async_tasks.py - Workload processing
server/website/website/tasks/periodic_tasks.py - Task scheduling
server/website/website/celery.py - Service settings
server/website/website/settings/common.py
Learning and analysis
- Data preprocessing
server/analysis/preprocessing.py - Workload clustering
server/analysis/cluster.py - Gaussian processes
server/analysis/gp_tf.py - Neural networks
server/analysis/nn_tf.py - DDPG policy
server/analysis/ddpg/ddpg.py - GPR models
server/analysis/gpr/gpr_models.py
Other components
- DB operator
- Target DBMS
Connections 23
- DB operator → Controller runner: runs
client/controller/src/main/java/com/controller/Main.java - Controller runner → Controller config: loads
client/controller/src/main/java/com/controller/Main.java - Controller runner → DB collectors: selects
client/controller/src/main/java/com/controller/Main.java - DB collectors → Target DBMS: collects from
- Controller runner → Result uploader: uploads results
client/controller/src/main/java/com/controller/Main.java - Controller runner → JSON utilities: parses config
client/controller/src/main/java/com/controller/Main.java - Result uploader → Service views: submits observations
- Service views → Tuning records: uses records
server/website/website/views.py - Service views → DBMS parser dispatch: parses DBMS data
server/website/website/views.py - DBMS parser dispatch → Vendor parsers: dispatches by DBMS
server/website/website/db/parser.py - Vendor parsers → Base DBMS parser: extends parser
server/website/website/db/myrocks/parser.py - Workload processing → Tuning records: aggregates results
server/website/website/tasks/periodic_tasks.py - Workload processing → Data preprocessing: uses preprocessing
server/website/website/tasks/periodic_tasks.py - Workload processing → Workload clustering: uses clustering
server/website/website/tasks/periodic_tasks.py - Workload processing → Task scheduling: uses task app
server/website/website/tasks/periodic_tasks.py - Recommendation tasks → Tuning records: reads tuning data
server/website/website/tasks/async_tasks.py - Recommendation tasks → Data preprocessing: cleans observations
server/website/website/tasks/async_tasks.py - Recommendation tasks → Gaussian processes: uses model
server/website/website/tasks/async_tasks.py - Recommendation tasks → Neural networks: uses model
server/website/website/tasks/async_tasks.py - Recommendation tasks → DDPG policy: uses policy
server/website/website/tasks/async_tasks.py - Recommendation tasks → GPR models: uses models
server/website/website/tasks/async_tasks.py - Recommendation tasks → Task scheduling: uses task app
server/website/website/tasks/async_tasks.py - Service settings → Tuning records: configures service