Giter VIP home page Giter VIP logo

frostfs-sdk-go's Introduction

frostfs-sdk-go

Go implementation of FrostFS SDK. It contains high-level version-independent wrappers for structures from frostfs-api-go as well as helper functions for simplifying node/dApp implementations.

Repository structure

accounting

Contains fixed-point Decimal type for performing balance calculations.

eacl

Contains Extended ACL types for fine-grained access control. There is also a reference implementation of checking algorithm which is used in FrostFS node.

checksum

Contains Checksum type encapsulating checksum as well as it's kind. Currently Sha256 and Tillich-Zemor hashsum are in use.

owner

owner.ID type represents single account interacting with FrostFS. In v2 version of protocol it is just raw bytes behing base58-encoded address in Neo blockchain. Note that for historical reasons it contains version prefix and checksum in addition to script-hash.

token

Contains Bearer token type with several FrostFS-specific methods.

ns

In FrostFS there are 2 types of name resolution: DNS and NNS. NNS stands for Neo Name Service is just a contract deployed on a Neo blockchain. Basically, NNS is just a DNS-on-chain which can be used for resolving container nice-names as well as any other name in dApps. See our CoreDNS plugin for the example of how NNS can be integrated in DNS.

session

To help lightweight clients interact with FrostFS without sacrificing trust, FrostFS has a concept of session token. It is signed by client and allows any node with which a session is established to perform certain actions on behalf of the user.

client

Contains client for working with FrostFS.

var prmInit client.PrmInit
prmInit.SetDefaultPrivateKey(key) // private key for request signing
prmInit.ResolveFrostFSFailures() // enable erroneous status parsing

var c client.Client
c.Init(prmInit)

var prmDial client.PrmDial
prmDial.SetServerURI("grpcs://localhost:40005") // endpoint address

err := c.Dial(prmDial)
if err != nil {
    return
}
    
ctx, cancel := context.WithTimeout(context.Background(), 5 * time.Second)
defer cancel()

var prm client.PrmBalanceGet
prm.SetAccount(acc)

res, err := c.BalanceGet(ctx, prm)
if err != nil {
    return
}

fmt.Printf("Balance for %s: %v\n", acc, res.Amount())

Response status

In FrostFS every operation can fail on multiple levels, so a single error doesn't suffice, e.g. consider a case when object was put on 4 out of 5 replicas. Thus, all request execution details are contained in Status returned from every RPC call. dApp can inspect them if needed and perform any desired action. In the case above we may want to report these details to the user as well as retry an operation, possibly with different parameters. Status wire-format is extendable and each node can report any set of details it wants. The set of reserved status codes can be found in FrostFS API. There is also a client.PrmInit.ResolveFrostFSFailures() to seamlessly convert erroneous statuses into Go error type.

policy

Contains helpers allowing conversion of placing policy from/to JSON representation and SQL-like human-readable language.

p, _ := policy.Parse(`
    REP 2
    SELECT 6 FROM F
    FILTER StorageType EQ SSD AS F`)

// Convert parsed policy back to human-readable text and print.
println(strings.Join(policy.Encode(p), "\n"))

netmap

Contains CRUSH-like implementation of container node selection algorithm. Relevant details are described in this paper http://ceur-ws.org/Vol-2344/short10.pdf . Note that it can be outdated in some details.

netmap/json_tests subfolder contains language-agnostic tests for selection algorithm.

import (
    "github.com/TrueCloudLab/frostfs-sdk-go/netmap"
    "github.com/TrueCloudLab/frostfs-sdk-go/object"
)

func placementNodes(addr *object.Address, p *netmap.PlacementPolicy, frostfsNodes []netmap.NodeInfo) {
    // Convert list of nodes in FrostFS API format to the intermediate representation.
    nodes := netmap.NodesFromInfo(nodes)

    // Create new netmap (errors are skipped for the sake of clarity). 
    nm, _ := NewNetmap(nodes)

    // Calculate nodes of container.
    cn, _ := nm.GetContainerNodes(p, addr.ContainerID().ToV2().GetValue())

    // Return list of nodes for each replica to place object on in the order of priority.
    return nm.GetPlacementVectors(cn, addr.ObjectID().ToV2().GetValue())
}

pool

Simple pool for managing connections to FrostFS nodes.

acl, checksum, version, signature

Contain simple API wrappers.

logger

Wrapper over zap.Logger which is used across FrostFS codebase.

util

Utilities for working with signature-related code.

frostfs-sdk-go's People

Contributors

fyrchik avatar kirillovdenis avatar carpawell avatar alexvanin avatar mastersplinter01 avatar roman-khimov avatar acid-ant avatar dstepanov-yadro avatar elichin avatar cthulhu-rider avatar aprasolova avatar

Stargazers

Snegurochka avatar Maria Fedorova avatar  avatar  avatar  avatar  avatar Anatoly Bogatyrev avatar  avatar  avatar Andrey Lesnykh avatar  avatar Vladimir Avdeev avatar Dmitri Usov avatar Vlad K. avatar  avatar  avatar Aleksey Chetaev avatar  avatar Алексей avatar  avatar Vladimir avatar Stanislav Bogatyrev avatar

frostfs-sdk-go's Issues

Reorganize handleError logic in pool

We have a bit of a hacks in handleError after #11: split info error should be passed as a status code from FrostFS but returned as a generic error anyway.

Consider making changes in API and storage node to avoid having split info error in transport error layer.
Or consider making handleError function parsing only error type without working with any statuses, so it will be at least consistent.

Renew token before it expired

Pool component might renew token before it is expired. This way we can avoid some retry requests and possible test failures.

Panic in SDK functions

Currently we panic in many situations:

  1. We forget to set an obligatory parameter.
  2. Some attribute has unexpected value ((netmap.NodeInfo).Price() -- we expect to get it with FromV2 where an error is processed).
  3. Something else?

While I think (1) is ok, others are not IMO. Also in (1) we could provide obligatory parameters explicitly: hard-to-misuse API is the best. Another option is to have New with obligatory parameters and Set methods on it (private structure with public interface).

@alexvanin @KirillovDenis @carpawell @realloc @acid-ant

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.