Immediate download about transfer HOT 4 OPEN

If a large transfer was ran inline during the API request, it could cause the request to time out, therefore interrupting the download. Cron jobs should be allowed to run for a much longer time before they are killed.

If every download was started immediately, we could end up with too many running at once, if lots of users requested transfers at the same time. I would like to keep some form of queue for this reason. However, there is no reason to wait 5 minutes for the next cron job to be triggered before starting to process the queue, if nothing is running already.

NCDownloader works around the waiting time by using Aria2 as a download manager, however I would prefer to keep the Transfer app simple to install, by not requiring any additional software on the server. This seems like something which could be improved in the main code of Nextcloud instead. It would be good to get all queued jobs started as soon as possible, both from this app and from others. Perhaps a background daemon which is always running, and waits for another job when it has nothing to do.

from transfer.

I agree but I'm afraid that such a change at the core of NextCloud would take a long time :(
I'm trying to think about two other possibilities :

Case of a forced triggering of cron.php:

Wouldn't it be possible for the Transfer application to immediately (I'm not sure how) trigger the execution of a cron job by executing immediately by itself the php -f /var/www/nextcloud/cron.php system command?
That would reduce a lot the initial cron execution random delay.

Or:

Case of immediate download with a simple request:

Wouldn't it be possible otherwise to, in the admin settings:

• Have an option to activate an immediate "dumb" download as soon as a link is submitted
• Have also an option to set the maximum number of simultaneous direct "dumb" downloads allowed per user during an immediate download (for example maximum 2)?

from transfer.

Would it be possible to set a "small file size limit"? Then the job could start immediately and if the file download is completed within that limit, everything is fine. If not, the job is paused/terminated and continued/restarted by the cron job.

This would be really useful for pulling files like pdfs (device manuals etc.) into the nextcloud without waiting.

from transfer.

I had previously tried to implement this, but unfortunately a lot of websites don't allow us to determine the file size (via a HTTP HEAD request) without starting the download. For that reason many small files would end up having to be queued anyway.

I suppose we could begin the download every time and interrupt it when the file size is known to be above the limit.

from transfer.