Support split config across the pipeline platform about tfx HOT 4 CLOSED

Not sure if here or #20 is better to start the conversation.

From reading the code, it seems like the split information would be needed by CsvExampleGen.checkcache because the data may exist but if a new split is being specified, then you'd want to cache bust and run CsvExampleGen.exec to avoid re-using an old split with the wrong split percentage

from tfx.

There are several levels for custom config:

1. Supporting a different train/eval ratio (which is #20 talking about): I think this should be mostly local to example_gen classes and should be easier to implement;
2. Supporting different split names from hard-coded train/eval: this is what #21 is talking about and would affect all components dealing with examples.

We hope to publish a RFC/design doc soon about 2), which will require 1).

from tfx.

Some update: step 1 (supporting a different train/eval ratio) is done in #20. We don't have an estimate on next step yet, but please let us know if you are blocked by it and we will revisit.

from tfx.

Custom split is supported, check this guide

from tfx.