adi64 / hyrise Goto Github PK

Add functionality to propose removal of existing, but unneeded indices.

Queries initiated using the SQL Pipeline should also be cached. This raises the issue of a globally accessible Cache Mechanism as long as SQLOperator remains in the System as another way to initiate Queries.

The SQLPipeline exposes intermediate results (like LQPs ref #1) that must be cached together with the resulting Operator Tree. Otherwise the availability of intermediate results becomes optional and the SQLPipeline might operate in two modes either using the cache giving no guarantee about the availability of intermediate results or not using it (reading, might still write on miss) always providing intermediate results.

Consider hooking into the query cache or adding a signalling mechanism instead of extracting data from it. This also would add the benefit of seeing how often a query is executed / references (refs #8 ).

Right now, an IndexProposal does not contain any information on how useful this change will be. Add metrics to it that allow to compare different IndexProposals, e.g. based on how often the column to be indexed is accessed, compared to other columns.

The contents of the query caches do not tell us how often a query is ran. In order to take that into account, we need to source information on how often queries are ran in order to calculate some metrics that should be taken into account when proposing new indices.

The execution of an IndexTuner should be schedulable and this functionality is already provided by AbstractTask.

Also a Manager for TunerTasks (another Singleton?) seems indicated, that periodically issues new Tasks and keeps track of their behavior.

Right now, we're scanning the operator tree top-down. A post-order traversal would probably be more adequate here. While doing this, we could build some data structure that keeps information for every operator regarding table and column data access (so we can know what table a TableScan is actually running on).

The IndexTuner should extract relevant Columns from Logical Query Plans instead of Operator Trees.
Currently the Query Cache does not keep LQPs for a Query, so it must be adapted.

Add workload (queries) to the playground that can be ran multiple times in order to experience performance gains through suitable automatic index creation. Also add simple timing / benchmarking functionality.