ractf / andromeda Goto Github PK

unoriginal

This is a stabler version of the original flask challenge server. However, its currently too close to RACTF 2020 to make any significant changes.

The main issue is to do with how scaling is handled, currently its based on the amount of users on each challenge which has all sorts of problems in practice. We don't know if a user is still actually using a challenge, we could track websocket disconnects but this isn't reliable enough, the other option is to add a disconnect button but we'd be relying on users actually clicking it, we can't kill a challenge while a user is using it and even if the user has been disconnected by the integration in core, they might still be using the challenge. Scaling based on CPU/RAM would be a better option however it would be awkward with the current API.

Resets are also inherently flawed, we pretty much had two choices in handling them, we could guarantee a new container each time or we could just guarantee the user doesn't get the container they were just on. In the flask version, it guarantees a new container every time, however this could lead to users creating hundreds of containers. Andromeda currently assigns the user a new instance out of the pool of instances for that challenge but we really need to find a middle ground, the housekeeping goroutine could look at the amount of people resetting a challenge and start a new instance to move people onto.

Scalability of Andromeda itself is also a significant problem with this, the backend integration is currently only designed to handle one instances of challenge server, this means if the challenge server crashes, all the challenges go down, and if there's too much load for one server to handle, whether that's the amount of containers or the API requests, there's not a lot we can do. There is the option of load balancing by taking the modulus of the user id and assigning them a challenge server, and that would work assuming all server are always up, but this would only scale to a few servers at which point it would be using resources too inefficiently and would provide no real failover as theres not an easy way to know if Andromeda is up.

There's also no persistent store for the instance data, this used to be handled by Redis in the flask version however that was just a side effect of Redis being used for sharing data between threads, this isn't necessary in Go thus we've lost the ability to recover the instance data. However, it's not really necessary, Andromeda could be adapted to look at the existing containers on startup and rebuild some of the instance data off that, we'd lose user data and that might cause problems with the housekeeping goroutine, but that could be handled pretty easily so I'm not convinced this is really that much of a problem for the scale Andromeda can run at.