baloo 1.4 Works fine -- 1.5 Crashes on .tar.gz download about balooproxy HOT 4 CLOSED

the source code for 1.5 is on the gofiber branch. Sadly gofiber doesnt support streaming, hence files will first be fully downloaded into the proxys ram, causing crashes if the files are too big

I have been debating if i want to port the changes i made in 1.5 to the main branch too but since they're pretty major and i havent really actively been working on this, i havent done so yet

Ill look into the iframe stuff, tho it's always worked for me

from balooproxy.

Ah! I did not realize that streaming was not supported, that makes sense! How unfortunate though.

Oddly enough, I overlooked the multiple branches.

gofiber/fiber#1034

"Fiber/Fasthttp supports response streaming to the client using the SetBodyStreamWriter"

gofiber/fiber#1971

to stream response, you will need create a new fasthttp Response and pass it to fasthttp.Client in the 
proxy middleware and pipe new reponse's body to fiber.Ctx's response (newResponse.BodyWriteTo(*fiber.Ctx) 
or use the SetBodyStreamWritter with a buffer).

But currently, proxy middleware only use the response of fiber.Ctx (c.Response()), 
so with proxy middleware, you can neither do stream proxy, nor modify/multiplying stream body.

You will have to use you own http.Client or fasthttp.Client to send requests.

Thanks for specifying, given that, I probably would hold off on porting the changes.
I like both, 1.5 definitely has a performance boost and consumes less resources but given that's at the expense of site functionality-- I'm not sure if it's worth it.

Is there anything alternative to http/gofiber, seperate to that, are there other potentials that you've had ideas for, to increasing the ability to handle requests at a lower resource cost?

With my test, 1.4, it consumed nearly all resources 20-30k R/S (none bypassed) and though it is a weaker CPU, it consumed all CPU at even less.
I'm just trying to see what optimizations can be done for those worst case scenarios.
I also may implement functionality to cut off acceptance of any new requests, or kill the existing highest resource consuming requests, once RAM hits a certain utilization percentage, to prevent baloo from outright being killed.(Like with the 1.5 issue)

I'm not familiar but it sounds like -- it could be worth while to switch from http to fasthttp, though I'm not sure at the moment. Any insights into limitations, problems or reasons why it may not work? I may try it out if I have the time
(That is, if the root issue of streaming can be worked around -- sounds like it may be able to, just not with gofiber)

I'm using Firefox, the only reason I knew that the iframe wasn't showing due to the contents not being encoded was from a past project where I ran into the same issue.

from balooproxy.

https://github.com/41Baloo/balooProxy/tree/merge-gofiber-improvements (soon to be merged into main)

should now have the improvements of the gofiber branch. Sadly it's not possible to support streaming with gofiber, as it's only able to stream data itself but will still have to first download the stream from the backend completely (AFAIK)

from balooproxy.

Merged into main now

from balooproxy.