donders-institute / dr-tools Goto Github PK

I always get this error in Linux and Mac:

GOPATH=home/go GOOS=linux GO111MODULE=on go install --ldflags="-X 'github.com/Donders-Institute/dr-tools/internal/cmd/version.Version=master'" github.com/Donders-Institute/dr-tools/...
/bin/sh: go: command not found
make: *** [build] Error 127

Why does it require ''go'' to be already installed?

It'd be nice if by default put/mput would try to retry the upload on error. I know that there is a parameter for that, but I only learned about it after seeing the errors due to connection failure in the log for a massive transfer job.

A common use case for me is that I only need a subset of a data repository, but the data I need are spread out over the entire directory tree of the repo.

I would like to give a list of files to get and give a destination directory (i.e Subject05), and to have the result that the copied data would be placed in a directory tree that mirrors that of the data repository. This is a behavior similar to the --parents option in the GNU version of the cp command.

One can get the following error if the source for upload is a symbolic link.

[11h3m14s:52h56m52s]/project/3015046.06/bids/derivatives/fmriprep/sourcedata/freesurfer/sub-x001/surf/lh.pial error:file size /dccn/DSC_3015046.06_991/bids/derivatives/fmriprep/sourcedata/freesurfer/sub-x001/surf/lh.pial mis-match: 4768106 != 10

It would be great to have the possibility to 'safely' run a transfer on a partially transferred directory tree, e.g. if only a part was transferred, it would be great to be able to just rerun the upload command on the base directory and repocli would identify which files still need to be transferred, e.g. based on a comparison of the size/modification date of the source and target files.

When trying to transfer files larger than a few gigabytes (not sure where the exact threshold is, but maybe around 50GB?) then the transfer repeatedly fails, sometimes without an error message.

Often the uploads take a long time, and one needs to very frequently check whether they are finished. It would be great to add the possibility to send an email upon completion of the upload, maybe even with the collected text output during the upload.

The readme says: "The WebDAV path should always be in form of the absolute path (i.e. started with /); while the local path can be in a format recognized by the shell.". Relative paths (starting with ./) seem to work fine as WebDAV paths for me.

It would be more intuitive, if repocli first collected the count of all files/total amount of data to be transferred, and then correspondingly labelled the progressbar. Currently, one has little prediction of how long the upload will take before finishing.

When I used this repocli tool on a ubuntu server to download the meg narrative dataset, it failed to download some files.

But if I download directly from the website, the downloaded files are fine.

I used the repocli.x86_64 from the 0.8.0 release.

Hi,
I wonder if a command for finding the size of a folder uploaded to repocli could be implemented?
As well as another one for finding the total number of files within a folder.
I normally do this with df -h and ls | wc -l respectively on linux.
I think these would be useful functions when trying to compare client and server copies of the data.

Best regards,
Adrian.

If you enter repocli shell, type in a command (e.g., put) and then a wrong path, deleting it takes a few seconds per character.

Dear developer,
I would like to ask why I get these errors when I use put to upload data to a repository in /dcn and how to deal with them.

1. Error 500: error:cannot stat … at the repository: PROPFIND … : 500
2. Error 401: error:cannot write … to the repository: Authorize https://webdav.data.donders.ru.nl/: 401

I add here the complete outputs I copied from the errors text file.

1. Error 500
   /media/genzel/MD04_RAT_RIPPLE/rat/Rat_OS_Ephys/Rat_OS_Ephys_Rat13_344994/Rat_OS_Ephys_Rat13_344994_SD13_OR_SD_N_13-14_06_2019/test/2019-06-14_11-52-42_trial6/100_CH18_0.continuous error:cannot stat /dcn/DAC_626830_0009_541/OS_basic/Rat_OS_Ephys_Rat13_344994/Rat_OS_Ephys_Rat13_344994_SD13_OR_SD_N_13-14_06_2019/test/2019-06-14_11-52-42_trial6/100_CH18_0.continuous at the repository: PROPFIND /dcn/DAC_626830_0009_541/OS_basic/Rat_OS_Ephys_Rat13_344994/Rat_OS_Ephys_Rat13_344994_SD13_OR_SD_N_13-14_06_2019/test/2019-06-14_11-52-42_trial6/100_CH18_0.continuous: 500

2. Error 401
   /media/genzel/MD04_RAT_RIPPLE/rat/Rat_OS_Ephys/Rat_OS_Ephys_Rat13_344994/Rat_OS_Ephys_Rat13_344994_SD1_CON_14-13_05_2019/2019-05-13_14-41-35_trial5/100_CH56_0.continuous error:cannot write /dcn/DAC_626830_0009_541/OS_basic/Rat_OS_Ephys_Rat13_344994/Rat_OS_Ephys_Rat13_344994_SD1_CON_14-13_05_2019/2019-05-13_14-41-35_trial5/100_CH56_0.continuous to the repository: Authorize https://webdav.data.donders.ru.nl/: 401

Thanks in advance,
Adrian.

See https://rclone.org