Pre-processing of data through Substra about substra-documentation HOT 3 CLOSED

The pre-processing of data is a time-consuming job. It could be interesting to be able, in the Substra framework, to perform data pre-processing work and to store intermediate data.
However, this raises security issues in relation to this storage of intermediate data.
Data pre-processing is a subject to have in mind but it will not be addressed in the immediate future.

from substra-documentation.

An advantage to do pre-processing trough Substra is also to give more freedom to the data scientist to explore and test different possibilities.

What could be interesting too is to give general information about the database. For example, in a project with images, it could be useful to normalize the inputs but then the mean and the variance of the train dataset should be known.

from substra-documentation.

Closing stale issue.

from substra-documentation.