PBS spawns unlimited amount of goroutine about prebid-server HOT 5 OPEN

@linux019 - what library for goroutines pools are you proposing?

@zhongshixi has offered to provide a pointer to a potential solution.

@bsardo will coordinate a decision.

from prebid-server.

I had to put an RPS limit on the load balancer.

IMHO its good practice to use backpressure limiting layers in front of Prebid Server. This is the approach we use to avoid the situation described here.

I have no issue adding a goroutine limiting feature to PBS provided it doesn't add latency when unused due to either disabled (if we want to provide that option) or using a high limit value.

from prebid-server.

I think this is more about reducing the compute spent on mcall at normal usage.

Being able to deal better with traffic spikes would be a side effect.

There should be no need to add a library for this.

from prebid-server.

@bretg we don't need third party library, many of them are over complicated. There is a good implementation https://github.com/panjf2000/ants it can be taken as example

from prebid-server.

we use https://github.com/panjf2000/ants

it works very well in our system since it preallocate the resources for go routines you need. Some improvement we did

1. we have different ants pool in different parts of the system to make sure not all concurrent execution compete on the same pool.
2. you do not want to shoot your own foot by having strict limit on the number of go routines, you need to have a soft limit and have a capacity to allow it to grow otherwise your execution can be stuck waiting for a go routine to be available.

from prebid-server.