session fails silently with large inputs about aiohttp-session HOT 6 CLOSED

@TheRobotCarlson what is the 'failing' behavior in your case (does the request silently crash, is the cookie 'lost', or ...)?

I tried to reproduce this but for me it worked pretty well, my guess is that this is somehow a limit on the user-agent used (testing with Insomnia the above server returned a response, with a cookie and sending this cookie back in a followup request resulted in a response that took the contents of the cookie into account).

RFC 6265 seems to have a 'minimum' requirement for user-agents to support at least 4096 bytes for each cookie including name, content & attributes (see RFC 6265, section 6.1).

The resulting Fernet-encrypted string has a length of 4068 and adding the default cookie name this brings us to 4086 so theoretically a well-behaving user-agent shouldn't reject this cookie.

As a general suggestion regardless of this issue I think that if your payload reaches this size you should consider using one of the other session storages (i.e. redis) that aiohttp-session provides, if not for any other reason for performance (transferring 'large' amounts of data to an 'adjacent' redis is way cheaper than sending them for a roundtrip to the user for each request).

Edit 1

Spoke too soon, the RFC refers to the: cookies that they [user-agents] can store, this means that on top of the 4086 bytes transmitted for this cookie all the cookie attributes and their default values are added before calculating this minimum. So a user-agent implementing the minimum requirements would probably evict this cookie.

from aiohttp-session.

@panagiks 'failing' behavior for me is silently not updating the cookie. When I run the above in Chrome Version 89.0.4389.128, I get no errors or warnings raised in the backend and instead have a silent failure that results in the cookie not being updated with any of the other inputs. An error or warning that the cookie "might be too large" is what I would expect.

Easy to accidentally add too much to a payload as an application's data grows. I added a new field to an existing application and that pushed it over the limit, it seems. I am planning on migrating to using redis session storage as a result of this, but didn't realize my payload had gotten that large until this failure appeared and stopped my application from updating any cookies.

from aiohttp-session.

The backend (at least the aiohttp-session part of it) technically updates the cookie and sends it over to your user (if you observe the request on the developer console of Chrome or try it with an API tester the cookie should be transmitted), the browser is the one that chooses to ignore it.

Having said that I also see no issue with adding a warning log if someone tries to save a cookie larger than some size ('some' here is debatable because the RFC refers to the 'total' stored size after adding all the attributes and not the transferred size that the library actually knows about) but I think a better place to add this would be aiohttp itself (since this concerns all cookies, not just cookies created through aiohttp-session).

@Dreamsorcerer what do you think ?

from aiohttp-session.

Indeed, a warning if it's going to exceed the RFC might be useful during development.

As you say, this is a problem more in aiohttp, so better to open an issue there to discuss it.

Specifically, the warning would probably go in somewhere around:
https://github.com/aio-libs/aiohttp/blob/1a4126a2c0e52b331aa953288e844bac680290fc/aiohttp/helpers.py#L852

Could also include a check on the total number of cookies as per that section of the RFC (although much less likely to be exceeded).

from aiohttp-session.

It's not entirely clear exactly what we should be counting for that limit though. Name + value is easy, but what about attributes? Does that mean the name + value of each attribute, or just the value, or name + value + separator, or what?

from aiohttp-session.

This is being tracked in aio-libs/aiohttp#5634 and associated PR.

from aiohttp-session.