Describe size limitations about neofs-api HOT 5 OPEN

The problem with missing object header size limitation affects the Neo Oracle service. One of Oracle NeoFS request types is getting header, and large header may be a problem for the Oracle node. Even if it won't fit in MaxOracleResponseSize limit, the Oracle node will try to load the whole header into memory before the response creation which may cause OOM. Ref. neo-project/neo-modules#803.

from neofs-api.

Describing an existing limitation and adding a limit where it never existed are somewhat different. In some cases from above it's the first, but in some (like object header size) it's the second.

from neofs-api.

It should be noted (thanks to @notimetoname digging) that although there's no explicit limit from the neo-fs side, the header itself is restricted by a single gRPC package (at least ObjectHead operation as far as object replications during ObjectPut). From the server-side gRPC client, this constraint is set to be max int32 value by default, and this default isn't changed anywhere. From the client side this value even smaller and set to be 4MB by default, this default isn't changed anywhere inside the neo-fs as far. Thus, if using neofs-api-go client, the user can't get header with size more than 4MB.

Thus, within the scope of this issue it would also be nice to provide the ability to edit client/server gRPC package size constraints as an option.

For now, consider neo-go Oracle response limit problem, we can be sure that the node is safe even if it doesn't restrict the received header size and no OOM could happen beause we have this implisit gRPC package limit. However, the C# node still may face with this issue. @roman-khimov, what do you think?

from neofs-api.

There has to be some sane NeoFS protocol limit for it, simple as that. 4M is too much, we've got 64M limit for the payload for the record. We need to take a closer look at this, but 4K seems to be a nice limit to me, at least I'd expect to see some value of that order for it.

from neofs-api.

we can start with some small value 1-4K, so we'll be able to increase the limit if there is no other way

from neofs-api.