persistent file storage (ZENODO storage + DOI minting for requests) about recast-flask-frontend HOT 4 OPEN

another idea is using GitHub as storage, but not sure how difficult it is to sync back to server

from recast-flask-frontend.

@cbora, actually probably the nicest and easiest way would be it we would just hook up the request submission directly to ZENODO. In the control center we already have the response side hooked up to ZENODO and it would create a nice symmetry between request and response (also fixed DOIs for both)

see code example here:
https://github.com/recast-hep/recast-control-center-prototype/blob/master/recastcontrolcenter/backendtasks.py#L95

and actual API calls here:
https://github.com/recast-hep/recast-control-center-prototype/blob/master/recastcontrolcenter/backendtasks.py#L30

from recast-flask-frontend.

@lukasheinrich OK, I will switch to ZENODO if you think that's better. Though, I had already integrated AWS S3. Currently, I am looking into file naming strategy to avoid filenames collisions, and so far I think using uuid is the best solution. If you have a better one let me know.

from recast-flask-frontend.

we might still need local storage, since I'm not sure that the only storage we ever need is the request data, so it's good to have something additional to zenodo. Maybe the upload to zenodo can be done as an additional step after it was copied to amazon z3. Also, if we upload to Zenodo, we probably should have DOI fields in the database model for BasicRequests / PointRequests), should we have a new issue for that?

As for filenaming.. at least the request data should come with their unique request uuid (scan_request_id field / point_request_id). can't we use this to build a folder structure.

from recast-flask-frontend.