How to scale relay-server about paralus HOT 4 CLOSED

@heng4fun relay server supports connection scaling to agent depending on the load ( number of user connections ). here's a gist of it,

i) by default server maintains 10 connections with respective agents
ii) if the current streams goes beyond the threshold ( 400 ), it triggers an auto-scale request which increases the number of connections to agent ( max 30 connections, it does so temporarily to handle the user requests )
iii) relay server then continuously checks whether there are any requests to these scaled clients, if not for a predefined healing period it scales the connection down ( min 10 connections )

if above does not suffice or there is a need to scale relay servers itself, then relay peering sentry will kick in and here's a gist of it
i) relay peering service in paralus core facilitates the peer learning mechanism, all relays are hooked into this service long lived channels.
ii) when a relay misses the cluster-connection lookup, it will probe the service to fetch its peer information. The service, in turn, performs the survey and find the relay that can handle the cluster, then sends a reply
ii) once the relay learns about its peer, it upstreams the blocked request to the peer.

hope this adds some context around how relay scaling / user requests are handled.

from paralus.

In this case, looks like relay-server is crashing and coming back up, if you can share some logs around the failure right before it crashed it will be helpful to understand what might be going on.

from paralus.

Thank you very much for the reply. I am recently looking into the issue with the relay-server and gathering some relevant logs.

from paralus.

Thank you very much for the detailed explanation.
We have finally discovered that the issue lies with our proxy server and not the relay server.

from paralus.