noisepage-pilot's Introduction

NoisePage Pilot

This repository contains the pilot components for the NoisePage DBMS.

Quickstart

Install necessary packages.
- pip3 install --upgrade -r requirements.txt
List all the tasks.
- doit list
Select and run a doit task from the task list, e.g. doit action_recommendation. Task dependencies are executed automatically.

Sample Doit Tasks

The following is a list of some relevant doit tasks and their parameters:

Action Recommendation
- The following task will begin picking indexes: doit action_recommendation.
Behavior Modeling
- doit behavior_datagen runs behavior model training data generation using TScout, Benchbase and Postgres. Requires sudo permissions for TScout. Also performs training data differencing (subtracting child-plan costs).
- doit behavior_train trains, evaluates, and serializes models along with their evaluations and predictions. It depends on behavior_datagen having been run first.
- Configure data generation, model training, Benchbase, and Postgres in noisepage-pilot/config/behavior.
- Training data differencing and model training default to using the most recent experiment data.
- Additional behavior modeling documentation is available at noisepage-pilot/behavior/README.md.

Background

Self-Driving DBMS = Workload Forecasting + Behavior Modeling + Action Planning.
- Workload Forecasting: forecast folder.
- Modeling: WIP.
- Action Planning: action folder.
- See ¹, ² for more details.

References

Make Your Database System Dream of Electric Sheep: Towards Self-Driving Operation.

@article{pavlo21,
author = {Pavlo, Andrew and Butrovich, Matthew and Ma, Lin and Lim, Wan Shen and Menon, Prashanth and Van Aken, Dana and Zhang, William},
title = {Make Your Database System Dream of Electric Sheep: Towards Self-Driving Operation},
journal = {Proc. {VLDB} Endow.},
volume = {14},
number = {12},
pages = {3211--3221},
year = {2021},
url = {https://db.cs.cmu.edu/papers/2021/p3211-pavlo.pdf},
}

↩