Comments (5)
The problem with missing object header size limitation affects the Neo Oracle service. One of Oracle NeoFS request types is getting header, and large header may be a problem for the Oracle node. Even if it won't fit in MaxOracleResponseSize
limit, the Oracle node will try to load the whole header into memory before the response creation which may cause OOM. Ref. neo-project/neo-modules#803.
from neofs-api.
Describing an existing limitation and adding a limit where it never existed are somewhat different. In some cases from above it's the first, but in some (like object header size) it's the second.
from neofs-api.
It should be noted (thanks to @notimetoname digging) that although there's no explicit limit from the neo-fs side, the header itself is restricted by a single gRPC package (at least ObjectHead operation as far as object replications during ObjectPut). From the server-side gRPC client, this constraint is set to be max int32 value by default, and this default isn't changed anywhere. From the client side this value even smaller and set to be 4MB by default, this default isn't changed anywhere inside the neo-fs as far. Thus, if using neofs-api-go client, the user can't get header with size more than 4MB.
Thus, within the scope of this issue it would also be nice to provide the ability to edit client/server gRPC package size constraints as an option.
For now, consider neo-go Oracle response limit problem, we can be sure that the node is safe even if it doesn't restrict the received header size and no OOM could happen beause we have this implisit gRPC package limit. However, the C# node still may face with this issue. @roman-khimov, what do you think?
from neofs-api.
There has to be some sane NeoFS protocol limit for it, simple as that. 4M is too much, we've got 64M limit for the payload for the record. We need to take a closer look at this, but 4K seems to be a nice limit to me, at least I'd expect to see some value of that order for it.
from neofs-api.
we can start with some small value 1-4K, so we'll be able to increase the limit if there is no other way
from neofs-api.
Related Issues (20)
- API inconsistency in reputation
- Can't get used space via API
- Provide method to list all containers in the network
- Drop subnets
- Object header size limitation HOT 7
- Store large objects hashes in the link object payload HOT 4
- Make it easy to seek through large objects HOT 2
- Numeric search matches
- Bearer token doesn't have any issuer field inside
- Add `400 Bad Request`-like status code HOT 2
- Do not store system data in raw key-value attributes HOT 2
- getrangehash doesn't work with a bearer token created via cli HOT 2
- Case-insensitive EACL matching
- `ZERO_LENGTH` RANGE status code HOT 5
- Account-based EACL targets HOT 5
- Drop __NEOFS__TICK things
- Functionality provided by `__NEOFS__NETMAP*` X-headers seems unusable and unviable HOT 2
- Move "objects" to `object` dir
- Expand session token statuses
- Drop 4M exclusion for headers
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from neofs-api.