nspcc-dev / neofs-api Goto Github PK

Storage node should be able to announce multiple addresses because it can service multiple networks. We have to make sure that repeated string address = 2 [json_name = "address"]; in netmap.NodeInfo structure does not break compatibility.

In nspcc-dev/neofs-node#700 we introduced the support for escape character in node attributes strings. This has to be reflected in API spec as well.

Different protocol gateways use attributes with directory path values to emulate file system directory structure. To simplify gateways development and to have a common approach on this, we should consider a "COMMON_PREFIX" match type for object search queries.

In it's v1 state it mostly covers implementation-specific functionality. Should be removed from neofs-api and go to neofs-node and other node implementations.

Extended ACL stored in the contract storage as raw slice of bytes. We've decided to have version in all statically stored structures like object and container. Shall we also have version field within extended ACL?

Most of the NeoFS API structures can be serialized into JSON, but not all structures have corresponding json_name tags assigned.

json_name tags should be added and serialization procedure specified at least for:

• accounting.Decimal
• acl.BearerToken
• container.Container
• netmap.NodeInfo
• netmap.PlacementPolicy
• object.Object
• storagegroup.StorageGroup

And related structures

Error message transfer and handling is an important part of distributed decentralized information system. Transport protocol implementations provide error handling mechanisms, however they differ from each other:

• gRPC inherits google API error model that returns error code, string message and optional details (supported in limited gRPC implementations),
• Twirp provides error code, string message and string key-value map for meta data.

NeoFS API does not define transport layer. Therefore it is required to describe error handling on top of all transport implementations. To do that we should:

• define enum of error types (or take it from gRPC or twrip implementation),
• define error structure with error type and string details,
• use error structure in ResponseMetaHeader.

According to #72 (comment) we've decided to describe string representation of object fields in neofs-spec. However there are a lot of information about well-known headers there, for example. And I don't see any issues with specifying string format in this repository. Also it should be implemented in neofs-api-go as well, so it makes sense.

We already agreed on format for object ID, container ID and address. But we have plenty more fields that we can filter on search request:

• object ID (base58 encoded string)
• container ID (base58 encoded string)
• object address (two base58 encoded strings with / as a separator)
• owner ID (NEO wallet address)
• version
• payload hash
• homomorphic hash
• object type

Edit 1:

• split ID

Previously it was possible to manage internal NeoFS Account directly using NeoFS API. It's currently possible by calling NEP-5 internal balance Smart Contract, but in theory, not all users can do it.

Maybe it would be beneficial to add simple Account operations without direct Neo Blockchain interaction, helping to improve user experience.

To prevent theoretical replay attack, we need to have Sidechain Magic in NeoFS requests.

Currently we have 2 (?) ways to share a collection of objects in our container with somebody:

1. Update eACL with the rule including objects ID (or some attribute) and public key to access them with.
2. Create a new container and push everything you need there.

Both of these are sub-optimal. eACL rule based on attribute values is not really an option in (1) because objects are already in the system so the only thing we are left with is to enumerate object ids. (2) doesn't scale well.

In essense the problem is to have per-object access settings without overloading eACL. Also, "sharing an object" doesn't look like an operation which requires altering container-level eACL.

One of the solutions is to introduce a linking object (similar to symlink in a file-system), which can have it's own headers, but share payload with the object it links to. This way we can have a single eACL rule like allow to GET objects with attribute ShareWith=123 for this public key. To share an object we can only put a link with the needed attribute in the container. To revoke access, simply delete an object (this plays nicely with an expiration epoch). While it can also be done with the help of some pre-defined attribute, I believe it is worth to be reflected in the API.

Currently Object delete request returns nothing. If there would be an Address of the newly deleted object's Tombstone, it could be some indicator of successful deletion process start.

netmap.NetworkInfo() method should return some useful public information about whole network. Right now it should contain:

• current epoch number,
• sidechain magic number.

This RPC intended to expand in later releases.

This method will be used for #52 implementation.

See nspcc-dev/neofs-node#328

We need to switch from using UUID as ObjectID to having ObjectID as a commutative Hash of Payload and Headers.

Having UUID as ObejctID was continent at project's early stages, but in NeoFS objects are immutable and we need to reflect that in addressing scheme.

ObjectID should be moved out of Header structure to Object message top level. This also affects the way we create Tombstones and split objects.

To simplify decoding of NodeInfo structure inside netmap smart contract, the first field, also used as a storage key inside contract, has to be of the fixed length.

In latest neofs-api version Object structure consists of:

• System header;
• Extended headers;
• Payload.

However, the list of extended headers also includes system information.
In turn, extended headers are defined through protobuf oneof which complicates the declaration of a strict header structure.

Proposed changes:

• unify System and Extended headers in a single Header message;
• make Object to be Header and Payload pair;
• rename System header to Main;
• move all fields of the System header except ID, CID, OwnerID and PayloadLength to Extended header;
• replace CreationPoint message with numerical CreationEpoch;
• rename UserHeader to object Attribute;
• encapsulate the headers of the object splitting hierarchy in the Split header.

Neo3 Oracles need a way to check object's MIME content type, as described in neo-project/neo-modules#555.

The proposal is to add Content-Type as a well-known attribute.

Objects and containers have attributes, a pair of key-value strings. Key should be always set but what about value? It seems like it is possible to set attribute with empty value: some sort of a tag for object or container. However this small detail requires extra changes in implementations. For example some key-value storages that can be used as a back-end storage, can't set empty values. Therefore you'll have some issues in implementing fast secondary index for attributes.

We should either explicitly declare in neofs-api, that empty attribute values are valid or restrict them.

/cc @realloc

Empty search filter returns all available objects in container (regular + sg + tombstones). It should be explicitly specified in the docs. To work with user related objects use $Object:ROOT filter.

To implement nspcc-dev/neofs-node#234 there should be two well-known x-header keys with prefix.
These x-headers represent epoch of netmap to use for lookup and the depth of previous epoch lookup (storage node looks for objects in nodes of previous epoch container if it can't find it anywhere).

Define string representation of these x-header values.

Mention, that it is useful first of all for read operations.

When BearerToken.body.owner_id is not set, token is not limited to be used only by one OwnerID and can be used by any bearer.

After nspcc-dev/neo-go#2160 we can expand NetworkInfo message with details about side chain:

• Ms per block,
• ... ?

Also we can include NeoFS global configuration records as a list of key/value (both repeated bytes).
Keys are hex encoded strings and values ara raw data from netmap contract.

NOT_PRESENT match type ignores values and looks if object has specified key.

Filter: A NOT_PRESENT 

Object1(not matched)    Object2(not matched)     Object3(matched)   
A: One                  A:                       B: Two
B: Two                  B: Two                   C: Three
C: Three                C: Three                 D: Four

Data audit makes many object.Head calls and these calls should be lightweight. We can send it with main_only flag in request, but the answer contains ShortHeader structure and it lacks hashes inside. However homomorphic hashes are quite important for data audit.

We should update ShortHeader definition with extra fields for homomorphic and SHA256 hashes of object payload.

As an EQUAL match type, NOT_EQUAL should be matched when object has specified key attribute, but it's value is different from specified one.

Filter: A NOT_EQUAL One

Object1(not matched)    Object2(not matched)    Object3(matched)
A: One                  B: Two                  A: Three
B: Two                  C: Three                B: Two
C: Three                D: Four                 C: One

It may be useful to support multiple addresses in a single DELETE request to save traffic and decrease average RPS load on the system in case of mass object deletion.

NeoFS uses different key and signature types, but this knowledge is not reflected directly in protocol definitions, only in code.

Signature structure needs to have key type reference and signature algorithm reference.

All types of attribute keys must be unique and can't be repeated in the same entity.

• Containers with duplicated attribute keys must not be accepted by InnerRing on creation.
• Nodes with duplicated attribute keys can't be accepted to NetMap by InnerRing
• Objects with duplicated attribute keys must be considered invalid and not accepted in PUT operations

Native NFT support should be reconsidered. This issue contains brief changes that were required for implementation and all connected tasks.

Define well-known container attributes for NFT ownership transfer, including:

• NFT-Chain
• NFT-Type
• NFT-Address
• NFT-Options
• NFT-ID

GetExtendedACLResponse message contains eACL signature in it's body. This signature is approved by alphabet nodes and valid in the moment eACL was changed. However later NFT can be transferred to other owner. Therefore public key of the current NFT owner might be different from the public key in eACL signature of GetExtendedACLResponse. (#153)

Container listing is going to be broken because container contract can't update owner values as soon as we do not monitor token transfers from main chain. Therefore container contract needs separate index on containers with NFT support and storage nodes should lookup over all containers with NFT to find out actual owner at the moment of the listing request. (nspcc-dev/neofs-node#524)

Other connected tasks:

• update neofs-api-go (nspcc-dev/neofs-api-go#279)
• check NFT owner in inner ring (nspcc-dev/neofs-node#508)
• sue NFT owner in ACL checks (nspcc-dev/neofs-node#507)

Instead of using bytes type for Container ID, OwnerID and thing alike, it may be more convenient to define and use specific types for them.

In the future, this structure can be ported into messages to maintain format uniformity. Also, storing the identifier in a dedicated message will allow, if necessary, to expand it with additional information without losing backward compatibility within one version of the API.

There are several ways container session token can be provided to and by container service.

Include session token in container and EACL bodies.

Objects already has session token field in the header, so it makes sense to do the same way with containers and EACL. This way we won't change signatures of RPC services and contract methods.

Include session token in meta headers.

The other way to include session tokens in request and response meta headers. In this case we should update container contract signatures and make additional storage for session tokens there. Also responses do not have token field in meta header yet. It actually doesn't make much sense, because it is meta information and it is not represented in response structures. It is not the case for the session token.

Personally I would go with option (1), because it requires small changes (two extra fields) and it is robust.

Container attributes are key-value pairs of strings that define immutable characteristics of the container.

Object version can be used in low-level processing when storage works with stream of raw bytes, e.g. object storage takes slice of bytes, cuts version field with some offset and then chooses appropriate unmarshaler function. To do so we should have object version field in the beginning of byte stream.

Right now object version stored in the beginning of Header message. With incompatible protocol changes, header message may be moved inside Object message, so version offset may change between different protocol implementation.

To avoid this, Object message can store version as a first field. But we should store version in Header message anyway, so it may be confusing.

Other solution -- to have an agreement to never change fields before Header in Object message.

Inner ring nodes make data audit and produce audit results. These results are not transferred between nodes but saved in audit contract. Later settlement routines will transfer payments based on these results.

Audit result structure should contain:

• epoch number (consider fixed size integer),
• container ID,
• public key of inner ring node that produced audit result,
• list of storage group object IDs that passed PoR,
• list of storage group object IDs that failed PoR,
• amount of hits at PoP,
• amount of misses at PoP,
• amount of fails at PoP,
• list of storage node public keys that passed at least one PDP,
• list of storage node public keys that failed at least one PDP.

There is need to check if node is ready to serve requests and get basic information about it. Previously this functionality was in Healthcheck service.

We need to add some simple ping-pong functionality without implementation specifics. It could be NodeInfo structure request in netmap service.

Most of Accounting functionality is now served by Smart Contract and there is no need to use NeoFS API for things other then requesting Balance.

ACL filter defines these keys to filter objects (outside of user-defiend headers):

• version
• container ID
• owner ID
• creation epoch
• payload length
• payload hash
• object type
• homomorphic hash

It mostly copies object search filters, however not all of them are quite applicable for ACL tables. Say there is no point to make ACL rules based on hashes or object type (ACL rules seems legit for regular objects only). Maybe we should short this list?

Since container API is a simple proxy between client and morph chain, there is no need in verification and meta header. They are basically unused.

Network Map is now managed by Smart Contract, so Node bootstrap process is now technically just a Smart Contract call. We no longer need to maintain this functionality in NeoFS API, and only basic Node State and Network Map requests are left.

Additionally, it becomes logical to move netmap data structures definitions from a separate library into NeoFS API for the benefit of all NeoFS Node implementations.

Container operations are similar to object operations in terms of permission delegation. Protocol gate implementations require permission delegation to operate with data on behalf of user.

Permission delegation is done by session token. We should expand session token body with container context. Container context includes:

• verb (Put, Delete, SetEACL),
• container ID.

We ignore all reading verbs because data publicly available on side chain, therefore it is useless to grant access for such operations.

There is a special container ID JEKNVnkbo3jma5nREBBJCDoXFVeKkD56V3xKrvRmWxFG (32 bytes of 0xFF) that should be interpreted as a wild card for the node. Node with delegated permissions can do specified operation with any user's container.

Put and Delete requests should have RFC-6979 signature, so smart-contract can validate requests from neofs-node. This way malicious node can't create unasked containers on behalf of owner. Put request should also have a public key, so container smart-contract can save it in neofsid contract for next signature verification.

Also it is more convenient to use container structure inside put request message. Fill structure, sign structure, put it into request, send request.

netmap.Operation enumerates operators between filters (OR, EQ, etc.). I think Operation term is more suitable for expressions like f1 AND f2 , so I suggest renaming to Operator.

Note: renaming of the message won't break version compatibility.

Storage nodes should be able to exchange reputation metrics about service consumers. This should be done via RPC defined in API. Reputation metric contains:

• interaction type,
• consumer public key,
• epoch,
• reputation delta,
• provider \ consumer flag.

GetExtendedACLResponse message contains eACL signature in it's body. This signature is approved by alphabet nodes and valid in the moment eACL was changed. However later NFT can be transferred to other owner. Therefore public key of the current NFT owner might be different from the public key in eACL signature of GetExtendedACLResponse.

NeoFS API protobuf definitions should be language agnostic. Usage of gogoproto plugin, which is Go specific, breaks this guideline. Hence, we need to stop using it in neofs-api.

Additionally, we could switch to lowecase field naming, as recommended in style guide.

Extended ACL description lacks objectID key in filters. While it makes no sense to use such filter in object.Search queries, it is kinda useful in extended ACL to grant or permit access to specific object.

I think it should be $Object:objectID according to JSON field name:

message Object {
  // Object's unique identifier.
  neo.fs.v2.refs.ObjectID object_id = 1 [json_name = "objectID"];

There is no clear statement on the maximal sizes of object header, attribute, eACL records, etc. It may be important for different language implementations.

EACL and SetEACL requests process Extended ACL as a slice of bytes, which is not clear for clients. API already defines EACLTable message with eacl rules, so it is convenient to use it in those requests.