dbiir / rainbow Goto Github PK

No document for detailed configuration properties of Rainbow.

The commands call receiver.action() when receiver is null.

Orc file format is not supported in Rainbow-evaluation;
OrcMetadataStat does not work when orc.compression is not NONE.

no MetadataStat for orc and other formats

There is no sql generation command under layout module.

Core module implements invokers and receivers which are used to construct a framework with all the other modules working as components in it.

1. .iml files and .idea/ directory are auto generated by Intellij IDEA, which are not needed in the codebase.
2. MANIFEST.MF file is auto generated during Jar file packaging, and should be excluded in the codebase.

An adaptive optimizer helps optimize one aspect (e.g. row group size or column order) of HDFS wide tables adaptively. There are three type of adaptive optimizer:
Column Order Optimizer;
Row Group Size Optimizer;
Compression Codec Optimizer.

We are going to implement at least the first two in V2.0. Compression Codec Optimizer can be left to a later version.

Once APC is true, do the follow steps:

• Add the optimization task into the optimization-queue(OP)
• Get several workload files and optimized layouts
• Watch the estimation charts
• Accept the suitable layout strategy

• Client module upload workloads regularly
• Web module receive workloads, and begin optimization adaptively
• Update the info of Pipeline

commands do not have uniformed params format and results format

Implement a Web-based user interface, including the front-end and the back-end.
The Web UI will interact with existing Rainbow modules to provide a user friendly layout optimization service.

no progress information when commands are executing.

The redirect module in the paper has not been implemented with Java.

no command for query redirect

The execute function of both CmdDuplicate and CmdOrdering under layout module is not implemented.

Readmes of evaluate and seek are not ready

Rainbow configuration file can not be specified in Rainbow-seek module.
Unit tests for this module are not enough.