DockerRunner performance improvement about clsi HOT 3 CLOSED

Hello @ShaneKilkelly,

thank you for bringing this up with the team!

There is already logic in place to prevent two compile requests from running in parallel on the same project/project+user directory 1. The other commands do not create new files and should be safe to run without any additional locking.
What's left is the creation and usage of container which need to be guarded from the container garbage collector. Note that #141 is important in this context. A parallel creation can be detected in the status code of 409 2, parallel calls on container.start result in a 304 3 - there is no need for a lock here.

In order to support the switching of the texlive image and more broader container option changes, I would suggest to keep the suffix in the container name. A hash of the container options, but without the command now.

The command timeout and kill logic both need some tweaking to support the reuse of container.
The timeout is currently implemented in JS with a setTimeout handler and a container.kill call. We could use timeout(1) from coreutils 4 instead. It signals a passed timeout with an exit code of 124 and forwards any exit codes on the happy route.
The kill logic is used only by the compile request? We could keep track of the Docker exec instance ID 5 which can be used to query the PID of the spawned process 6.

Now that I finished my studies - yay - my sharelatex setup is just under synthetic load and serving as a distributed system to experiment/work with. I will probably see no performance impact from this, but I will observe slightly better response times on paper 😉 and eventually notice it when working on the frontend/editor.

from clsi.

Hi! Thanks for doing the investigation on this, it's nice to see some concrete numbers!

We've had a quick discussion with the team and decided that the reason we didn't go for this pattern was that it would have it's own resource trade-offs (keeping containers alive in an idle-loop even while they're not necessarily active).

At the moment we have logic to re-use an existing container if it's present on the system, so the initial cold-start cost should only be paid rarely, and most actions should be relatively fast (as you've discovered).

Another thing that comes to mind, which would need to be investigated, is the kind of locking and synchronization logic that might need to be introduced to prevent overlapping commands from clobbering each other in the one container.

I think we'll park this for the time being and keep it in mind for optimizing our production load in the future. In the meantime, do you have a sense of how this would change the performance characteristics on your own system under normal workloads?

from clsi.

Hi!

Thank you for taking the time to write up this issue.

We are in the process of migrating to a monorepo at https://github.com/overleaf/overleaf and will mark this repository read-only soon.
You can read more about the monorepo migration at overleaf/overleaf#923.

We are going to close this issue now to avoid any confusion about the inability to comment further.

If you believe this issue still needs addressing, please create a new issue at https://github.com/overleaf/overleaf.

Thanks again!

from clsi.