HTTP Requests from Chrome block the server about adafruit_circuitpython_httpserver HOT 13 CLOSED

Sorry, should have given my board/version:

Adafruit CircuitPython 8.0.0-beta.6 on 2022-12-21; Adafruit Feather ESP32S3 4MB Flash 2MB PSRAM with ESP32S3

from adafruit_circuitpython_httpserver.

Can you show the request.headers Chrome is sending? Even better, the whole request: request.header_body_bytes or request.raw_request.

from adafruit_circuitpython_httpserver.

Sure! Here is Chrome:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cache-Control: max-age=0
Connection: keep-alive
Host: 192.168.99.6
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36

and to make the point I think it is Chrome specific - here is the same request in Firefox:

Host: 192.168.99.6
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:106.0) Gecko/20100101 Firefox/106.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1

from adafruit_circuitpython_httpserver.

I tried the code you provided on ESP32-S2 TFT as it is the only WiFi board I have, works fine. Have you tried it on any other board?

EDIT 1: During my refactor and revamp of this library I was using Chrome for the whole time, never experienced something like that, I wonder whether it is board specific.

from adafruit_circuitpython_httpserver.

Thanks so much for trying 👍
No I have not tried it on any other board (dont have another) happy to buy another one and try it ...

from adafruit_circuitpython_httpserver.

There are some things that come to my mind as the probable reason for that behavior, mostly browser related.
The key is to find an exact line in code that blocks.

If it is ok for you we can try to find a cause of this on Discord call.

from adafruit_circuitpython_httpserver.

There are some things that come to my mind as the probable reason for that behavior, mostly browser related. The key is to find an exact line in code that blocks.

If it is ok for you we can try to find a cause of this on Discord call.

Sure happy to do that - bear in mind that it blocks even if you hit a non-existent url (i.e. no obvious line of code executes?)
Just about to meet a pal (UK time - hence beer o'clock) but if you let me know how to do it I can do a discord call tomorrow?

EDIT: If you want to send me anything to try, happy to do it first thing tomorrow

from adafruit_circuitpython_httpserver.

By line of code I also meant the line in library code.

I'm located in Poland so there is only 1h diffrence between us. Tomorrow works for me, I'm flexible with time.
My Discord ******, please add/message me so we can schedule a call for tomorrow.

EDIT: If you want to send me anything to try, happy to do it first thing tomorrow

EDIT 1: First thing I would do is placing some print statements in adafruit_httpserver/server.py, you probably have a .mpy file so debugging it is not really possible, try downloading a .py version of library. Using these prints it should be possible to pinpoint where the block occurs.

from adafruit_circuitpython_httpserver.

EDIT 1: First thing I would do is placing some print statements in adafruit_httpserver/server.py, you probably have a .mpy file so debugging it is not really possible, try downloading a .py version of library. Using these prints it should be possible to pinpoint where the block occurs.

So I did as you suggested and sprinkled print statements into server.py, the board pauses for 60 seconds in the _receive_header_bytes() function, specifically, during sock.recv_into() ...

    def _receive_header_bytes(
        self, sock: Union["SocketPool.Socket", "socket.socket"]
    ) -> bytes:
        print("Receive Header Bytes")
        """Receive bytes until a empty line is received."""
        received_bytes = bytes()
        while b"\r\n\r\n" not in received_bytes:
            try:
                print("Try Receive Header Bytes")
                length = sock.recv_into(self._buffer, len(self._buffer)) <------ pauses here
                print("Received Header Bytes")
                received_bytes += self._buffer[:length]
            except OSError as ex:
                print("Exception OS")
                if ex.errno == ETIMEDOUT:
                    break
            except Exception as ex:
                print("Exception Other")
                raise ex
        return received_bytes

I noticed that even with a request from Firefox, the code can linger there for a few seconds, but it does not pause the way that Chrome does.

EDIT: After further investigation, it seems that Chrome makes the request, receives the result and them immediately opens another socket ready to send another request, BUT does not send anything down this socket connection, unless of course you make a Chrome request whereby it uses that connection to send the data, receives it and again opens another unused connection. After 60 seconds, the open connection is closed (not sure whether it's closed by Chrome or the server) and everything continues.

from adafruit_circuitpython_httpserver.

Together with @yousafs we did some testing and came to the following conclusion:

• This problem does not appear on CircuitPython 7.3.3, only on 8.0.0
• Problem exists on both ESP32-S2 and ESP32-S3
• The core problem seems to be connected to probably incorrect behaviour of timeout on socket, despite setting it it is simply ignored, which causes blocking
• On CP 7.3.3 after processing request program loops .pool(), on CP 8.0.0 most of the time it gets stuck on sock.recv_into()
• Serial connection seems to be blocked when stuck on sock.recv_into()

The problem seems not to be caused by the adafruit_httpserver library, but by the issue in socketpool, probably in .settimeout(). Separate issue will be opened on CP's repository.

from adafruit_circuitpython_httpserver.

Thanks @michalpokusa.
FWIW I think this will be a more robust fix across a range of feathers, rather than relying on the underlying socket library being fully implemented on every board.

from adafruit_circuitpython_httpserver.

Following the release of the stable CP 8.0.0 I tested whether this issue is still present.
Blocing is no longer the issue, the adafruit/circuitpython#7455 fixed it a while ago, thanks @dhalbert!

I believe this issue can be closed as resolved.

from adafruit_circuitpython_httpserver.

@yousafs This should be resolved, you can re-open if you still have the issue.

from adafruit_circuitpython_httpserver.