Responses contain `Content-Length` header and `Transfer-Encoding:chunked` headers about sharry HOT 21 CLOSED

Can you be more specific? I updated my installation and everything seems fine here - so far at least. I can access the old stuff. Are you accessing files as an authenticated user or via a published share?

from sharry.

The shares are anonymous uploads to an alias.
I'm authenticated via a basic account, go to my share click on on of those share and Chrome fails, with curl I have:

* using HTTP/1.1
> GET /api/v2/sec/share/2pbk76iRTty-cjfji3fdGNs-uWGkiyEXY5C-xxxx/file/6xcKgvcHDc1-rVd8UAqhMVw-BUhKMmVa7DX-xxxxxx HTTP/1.1


< HTTP/1.1 403 Forbidden
"Authentication failed."

I was working with the nightly build after the merge of my PR.

And reverting to v1.12.1 fixes this too. Unfortunately you do not publish snapshots for each commits so can't pinpoint more the problematic commit.

from sharry.

Strange, I can't reproduce this. I tested with previous and new uploads to an alias. With the curl command, did you add the cookie or Sharry-Auth header? Otherwise the response is expected and I can't see the exact request from what you posted.

Can you perhaps check chrome's inspector what the requests look like in the failing case? Between your commit and the current state, there are only very few commits - none of them really change things near this from what I can see.

from sharry.

Ok so doing more tests it seems that there's a change that breaks Cloudflare in front.

The sharry docker logs shows it return 200

2024.01.07 10:55:23:0000 [io-comp...] [INFO ] org.http4s.server.middleware.Logger - HTTP/1.0 GET /api/v2/sec/share/Cc3WSr45mCG-NXcRyWmHDuY-8t14WEZz75t-xxxx/file/6d4GvysLRa5-mQSQSXEkJKH-h979YvwQK7X-xxxxx
2024.01.07 10:55:23:0001 [io-comp...] [INFO ] org.http4s.server.middleware.Logger - HTTP/1.1 200 OK

But Cloudflare returns 502

So it might be the change in proxy auth / headers that does something strange with Cloudflare.
Maybe there's a new setting I need to change?

from sharry.

Hm, to my knowledge there is no change necessary. Considering it worked for you with the nightly after the merge of the description search it is very strange to me. Between 1.12.1 and 1.13.0 there have been lots of library updates that might have had changes for parsing http requests - but I don't think it is very likely though. Between your commit and 1.13.0 there is only this proxy-auth change which adds another way (another route) to create an auth token, but doesn't touch anything else - i.e. not getting shares. The diff: a42bdc2...v1.13.0

I don't know cloudflare, so I can't help much here. What kind of changes could cause such a behavior from cloudflare?

from sharry.

I don't know, Cloudflare is just a proxy that hide the server real address and add https certs easily + free cache.

When passing the cookie via curl it fails with 403 but the logs shows that too

2024.01.07 11:23:32:0000 [io-comp...] [INFO ] org.http4s.server.middleware.Logger - HTTP/1.0 GET /api/v2/sec/share/2pbk76iRTty-cjfji3fdGNs-uWGkiyEXY5C-xxxxx/file/6xcKgvcHDc1-rVd8UAqhMVw-BUhKMmVa7DX-xxxxx
2024.01.07 11:23:32:0001 [io-comp...] [INFO ] org.http4s.server.middleware.Logger - HTTP/1.1 403 Forbidden

* schannel: disabled automatic use of client certificate
* ALPN: curl offers http/1.1
* ALPN: server accepted http/1.1
* using HTTP/1.1
> GET /api/v2/sec/share/2pbk76iRTty-cjfji3fdGNs-uWGkiyEXY5C-xxxx/file/6xcKgvcHDc1-rVd8UAqhMVw-BUhKMmVa7DX-xxxxx HTTP/1.1
> Host: xxxxx
> User-Agent: curl/8.4.0
> Accept: */*
> Cookie: sharry_auth=1704624846002-QjlMUkNVYUFOZmYtS0M1Uzh5ZXZwd3MtWkFza1dFYnljb3UtMzVxxxxxxxxxxxx=
>
* schannel: remote party requests renegotiation
* schannel: renegotiating SSL/TLS connection
* schannel: SSL/TLS connection renegotiated
< HTTP/1.1 403 Forbidden
< Date: Sun, 07 Jan 2024 11:23:32 GMT
< Content-Type: application/json
< Content-Length: 24
< Connection: keep-alive
< CF-Cache-Status: DYNAMIC
< Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=xxxxxxxx"}],"group":"cf-nel","max_age":604800}
< NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
< Server: cloudflare
< CF-RAY: 841bd44ssss-MRS
< alt-svc: h3=":443"; ma=86400
<
"Authentication failed."* Connection #0 to host xxxxxx left intact

I'm lost at what is happening here :(

from sharry.

Really strange. It could be that the proxy maybe removes some headers? There have been problems with cloudflare reported in the past (for example #631 and #1063) - not sure how that was solved. On my other project, someone had similar problems (eikek/docspell#1717 (comment)).
It is really weird that it works with an earlier version, that could be related to caching. If it helps I could create another zip package from some other commit that you can try (you could also do it yourself, it's sbt make-zip). Not sure how to best proceed.

from sharry.

The strange thing is error 502 and not auth error, so seems more something that happens after the file start downloading since in Sharry logs the get says 200.

On the prod server I can only test docker images but if you can generate docker images on docker hub then I can make tests with more logs yes.

from sharry.

Ah, so the share list view renders fine? And then you click on a share to go to the details - this is also good? The error then happens once you download a file?

I don't think I can easily create docker images for any commit right now :/

from sharry.

Yes the error is about downloading a file, all the rest works, anonymous uploads, listing, ...

But this is quite problematic as the main end purpose of this :p

from sharry.

Ah ok, I thought it is getting the share (looked like from the first comments/description). That code was not touched at all.

But tbh, I don't know what the cause could be. I cannot reproduce it here. I still think something is removing request headers? Downloading a file is guarded by the same code that guards the other routes (looking from an auth perspective).

When downloading a file, there has been one small change: 4617439

I can't see how that would mess with cloudflare in any way, but also, as I said above, I don't know cloudflare (I mean I know what it is, but not much more than that :-)). It fixes the response by setting a correct ETag header and returns now (hopefully) a correct range response. It also fixes the If-None-Match check. That code would be used when downloading any file (from published shares or private ones) - perhaps you could try to download a file from a public share - if that doesn't work as well, the change here could be related. In my case(s) I can't see any errors… (using nginx as a reverse proxy and without any reverse proxy).

from sharry.

I tried to publish the share then copy link and open from that it does not work too :(

from sharry.

If cloudflare responds with a 502 it usually means that the upstream server responded with some error. Can you see what happens at sharry when you download a file? Perhaps also the request headers that are passed to sharry? Given that the download of a public share fails, it is very likely not related to any auth thingy as we thougt first.

from sharry.

The thing is that sharry only logs shows it works

HTTP/1.0 GET /api/v2/sec/share/5J7cWx5KMdR-HzUWhLkFjFw-4LYdsC8aRMj-g6q1AF8DTvp/file/9rnq7PoqaFz-szFPBJb8zoJ-E1WXKuoKCVf-xxxx
2024.01.10 07:10:55:0001 [io-comp...] [INFO ] org.http4s.server.middleware.Logger - HTTP/1.1 200 OK

Attached an redacted HAR for the download from a public share.

sharry.zip

from sharry.

I looked at the har file, but can't see anything useful there. 😞 CF says "bad gateway", from what I know this means sharry responds with something erroneous. Perhaps you can try and set the logging level in the config to more than info (sharry.restserver.logging.levels.org.http4s="Trace" and sharry.restserver.logging.levels.sharry="Trace" for example). Another option could be that CF expects something that it didn't get (response headers?)?

from sharry.

So removed cloudflare and left only nginx and nginx returns 502 bad gateway too :(

Enabled both logs at trace level does not output anything more.

sharry       | 2024.01.16 12:15:36:0000 [io-comp...] [INFO ] org.http4s.server.middleware.Logger - HTTP/1.0 GET /api/v2/sec/share/DgzXRR7Nevg-bhp46z9SkoN-YtLoBAdvzQi-ctnW2jA2aYB/file/GyDq36u3ufg-ueWjuB                                                                             eueue-xxxx-xxxx
sharry       | 2024.01.16 12:15:36:0001 [io-comp...] [INFO ] org.http4s.server.middleware.Logger - HTTP/1.1 200 OK

Sharry says 200 but fails after.

from sharry.

@eikek So after tons of debugging the error is:

upstream sent "Content-Length" and "Transfer-Encoding" headers at the same time while reading response header from upstream

Seems this is invalid and proxies will reject that. Don't know what causes this but we have a lead now;

from sharry.

Oh wow, thank you very much! Very strange, not everyone is rejecting this probably - strange that I don't see it in my installation. I'll check where that sneaked in.

from sharry.

Hi @Tolriq I added a change to master that should remove the Content-Length header. I think most clients just ignore it for chunked responses. It has been like this forever, it is still a bit strange that you ran into this problem now. I hope it solves the issues - would be great if you could try out once the ci has went through everything.

from sharry.

@eikek Thanks a lot it works now :) I can finally search in description.

from sharry.

Thanks for testing! I'm glad it's working.

from sharry.