Giter VIP home page Giter VIP logo

cothority's Introduction

Build Status Coverage Status

Navigation: DEDIS :: Cothority

This repository is currently maintained by C4DT.

Cothority

The collective authority (cothority) project provides a framework for development, analysis, and deployment of decentralized, distributed (cryptographic) protocols. A given set of servers running these protocols is referred to as a collective authority or cothority. Individual servers are called cothority servers or conodes. The code in this repository allows you to access the services of a cothority and/or run your own conode. The cothority project is developed and maintained by the DEDIS lab at EPFL. To read more about cothorites you can have a look at the following paper.

This is an overview of this README:

  • Documentation with links to different parts of the cothority
    • Topically ordered explains the different functional pieces from the cothority from a research point of view
    • Network ordered gives an overview from a networking point of view
    • Links collects links to other repositories, papers and webpages that are important to the cothority.
  • 1st steps giving two example apps ready to use
  • Participating on how to help us getting cothority even better

Don't forget that the cothority is part of a bigger environment.

Versioning and Roadmap

The cothority repository is currently being maintained by the factory team of https://c4dt.org. We're mostly following semantic versioning, but since DEDIS is not adding new features to the project anymore, we suppose that the major version 3 is the last one. Which means that we're trying hard not to break backward-compatibility , except for currently unstable features like BEVM. For new features, please wait for https://github.com/dedis/dela to stabilize.

The Factory team at C4DT has the following plans to be implemented in the cothority, which mostly concentrates on ByzCoin:

The nodes running at https://status.dedis.ch do not use any versioning, but are rather deployed once a day using the latest code. This is done in https://github.com/c4dt/byzcoin.

For the typescript libraries, we do use semantic versioning, as they are included by external apps. Please have a look at PUBLISH.md to see how that works.

The exception to the above are the e-voting binaries which are published by tagging a release in this repository. Please have a look at PUBLISH.md for information on creating a release for e-voting.

Release v3.1.0

The release introduces the notion of signature scheme for a given skipchain so that one can define which co-signing algorithm will be used to sign the forward links. This was necessary in the context of weaknesses in the BLS signature algorithm (see the paper). New skipchains will be created with BDN set as the signature scheme.

Because of a new scheme is default, that means that skipchains created after v3.1.0 won't work with older versions as they are not aware of the new scheme. However, existing skipchains will continue to operate normally. In summary, if you need to create skipchains after updating to v3.1.0, make sure every conode is at least using v3.0.1 aswell.

Release v3.2.0

A new field has been added to the DataHeader, Version, so that new features or upgrades can be coordinated between the conodes to only start using it when enough of them are up to date. The leader will propose a change of version when it detects that enough of the participants can reach a consensus. A successful increase of version is announced by an empty block that will act as a barrier between the previous and the new version. Its DataHeader data will contain the new version.

When creating a ledger, the default version is the most recent one and blocks are continously created with the previous block version until the leader proposes an upgrade. Note that the initial version is zero for backwards compatibility.

Another important change for this version is about how transactions are created as they need to include the ByzCoin version to use the correct hash function. The initial version of the hash was not taking the invoke command into account and it has been fixed for version one and higher. See below examples:

Go

client := byzcoin.NewClient(id, roster)
tx := client.CreateTransaction(instr1, instr2)

Java

ClientTransaction tx = new ClientTransaction(instrs, rpc.getProtocolVersion());

Javascript/Typescript

const tx = ClientTransaction.make(rpc.getProtocolVersion(), instr1, instr2);

Release v3.3.0

An experimental contract has been added to ByzCoin making it possible to use Ethereum contracts. See directory bevm.

The ByzCoin client-side API version number has changed from 1 to 2. Callers should use the new version in their requests, but the change is backwards compatible and old clients will still work.

Documentation

The goal of the cothority is to collect projects that handle aspects of collective authorities to support scalable, self-organizing communities. In this document we present the apps that are directly runnable from the cothority, as well as links to the services and protocols used in the cothority.

A cothority is a set of conodes that work together to offer collective authority services. This is used to create distributed and decentralized services where no single point of failure can put the whole system in jeopardy. Conodes communicate with each other using protocols that are short-lived, specific tasks with an outcome that can be read by services. Each conode has a set of services that can be contacted through a well-defined API from the outside. The communication through the API is done with a homebrewn protobuf over websockets implementation.

Topically ordered

When looking at the cothority modules from a topical point of view, we can break it up as follows:

+--------------------------+------------+--------------------------+
|                          |APPLICATIONS|                          |
|     Onchain-Secrets      +------------+     Proof of Personhood  |
|                                                                  |
|       ByzCoin                 Status            E-voting         |
+------------------------+---------------+-------------------------+
|                        |BUILDING BLOCKS|                         |
| Consensus              +---------------+       Key Sharding      |
|  - Skipchain                                    - Re-encryption  |
|  - BFT         General Crypto    Messaging      - Decryption     |
|  - Collective   - Neff Shuffle    - Broadcast   - Distributed    |
|    Signing      - RandHerd        - Propagate     Key Generation |
|                 - RandHound                                      |
|                                                                  |
+------------------------------------------------------------------+

Applications

Applications in cothority use different libraries and are the highest level of abstraction in our framework.

Here you get a list of all applications in the cothority.

There is one very special application that is considered apart - it's the conode itself, which holds all protocols and services and can be run as a service on a server.

What a Conode can do for you

Building Blocks

Building blocks are grouped in terms of research units and don't have a special representation in code. Also, the General Crypto library is more of a collection of different protocols.

Here you get a list of all building blocks in the cothority.

Network Ordered

If we look at the cothority from a purely networking point of view, we can describe it as follows:

              +-----------------+
              |CLI, JavaScript, | Frontend
              |Java             |
+-------------+-----------------+
| Conode,     | Services        | Client to Conode
| Simulations |-----------------+
|             | Protocols       | Conode to Conode
+-------------+-----------------+

Command Line Interfaces

Command line interfaces (CLI) are used to communicate with one or more conodes. All CLIs need to have one or more conodes installed. For the two CLIs in first steps, you can use the running conodes at EPFL/DEDIS. If you want to test the other CLIs, you might need to set up a small network (often 3 nodes is enough) of conodes on your local computer to test it.

Here you get a list of all available CLIs in the cothority.

Services

Every app communicates with one or more services to interact with one or more conodes. This is a list of available services that can be contacted on a running conode. Some of the services are open to all, while others might require authentication through a PIN or a public key. Most of the apps and services have the same name, but some are not available as an app or have more than one app using it.

Here you get a list of all available services in the cothority.

Protocols

Protocols are used to communicate between conodes and execute cryptographic exchanges. This can be to form a collective signature, create a consensus on a new state, or simply to propagate a new block from a skipchain. Some protocols are useful for different services, while others are very service-specific. Most of the protocols have a paper that is describing how the protocol should perform and that compares it to other protocols.

Here you get a list of all available protocols in the cothority.

Simulations

Cothority grew up as a research instrument, so one of its advantages is to have a framework to create simulations and running them locally or on remote servers. Some of the protocols presented here do have the simulation code. Check it out here: Cothority Simulations.

First steps

If you're just curious how things work, you can check the status of our test network or create a collective signature using our running nodes:

Status

To get the status of the conodes in the cothority, first install the status binary:

go install ./status

Now you can run it by giving the definition of the dedis-cothority on the command line:

status -g dedis-cothority.toml

Participating in the cothority

There are different ways to participate in the cothority project. A first step is to simply test the different CLI applications in this repository and tell us what were your difficulties or what you would like to use them for.

A next step is to set up your own conode and participate in consensus operations on skipchains or ledgers.

Setting up your own conode

A conode is a server program that includes all protocols and services. It can be run on a public facing server, but for testing purposes it's also possible to set up a network on a machine and only let it be accessed locally.

  • conode is the cothority server, a special app that includes all services and protocols.
  • How to run a conode gives an overview of the environment where a conode can be run
  • DEDIS-cothority explains how to join the DEDIS-cothority

Contributing

If you want to contribute to Cothority-ONet, please have a look at CONTRIBUTING for our coding practice and licensing details. In short, we use the github-issues to communicate and pull-requests to do code-review. Travis makes sure that everything goes smoothly. And we'd like to have good code-coverage.

License

The software in this repository is put under a dual-licensing scheme: In general all of the provided code is open source via GNU/AGPL 3.0, please see the LICENSE file for more details. If you would like to use Cothority in a way not allowed by the applicable license, please contact us to inquire about conditions to get a commercial license.

Contact

We are always happy to hear about your experiences with the cothority project. Feel free to contact us on our mailing list or by email.

Reporting security problems

This library is offered as-is, and without a guarantee. It will need an independent security review before it should be considered ready for use in security-critical applications. If you integrate Cothority into your application it is YOUR RESPONSIBILITY to arrange for that audit.

If you notice a possible security problem, please report it to [email protected].

Who is using our code?

This is a list of people outside of DEDIS who is using our codebase for research or applied projects. If you have an interesting project that you would like to have listed here, please contact us at [email protected].

  • Unlynx - A decentralized privacy-preserving data sharing tool
  • Medco - Privacy preserving medical data sharing
  • ByzGen - Tracking and secure storage of digital and hard assets
  • PDCi2b2 - Private Data Characterization for i2b2

cothority's People

Contributors

ayushman17 avatar cgrigis avatar christianmct avatar christophebertrand avatar claudio5 avatar daeinar avatar darekzbik avatar decltypeme avatar evanv03 avatar froelich avatar gilthoniel avatar gnarula avatar ilmarkov avatar ineiti avatar jeffallen avatar jiachengxu avatar kc1212 avatar keducoop avatar lefkok avatar liamsi avatar lmeynent avatar mlncn avatar nikkolasg avatar nkcr avatar paraacha avatar pegger avatar pierluca avatar tharvik avatar valentinmoullet avatar yvgny avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cothority's Issues

Naming

Can we use somewhat more concise directory/package names, when they'll still be clear but save a bit of typing and screen space? Suggestions:

application -> app
protocols -> proto
helpers -> lib?

Monitor: Discard n slowest measures

Because some of the runs take a lot more time in some special cases, we'd like to measure only the fastest measurements. So instead of only dropping the first measurement, drop the 'n' slowest measurements, where 'n' is a variable available on the command-line or in the configuration-script.

Use deploy with protected ssh-key

When the ssh-key is protected by a password, deploy can't connect to the users.deterlab.net-server without having to ask for the password all the time.

Use ssh-agent or any other convenient tool to bypass the password-request.

Deploy

Things noted in deploy that may need to be changed:

  • Deterlab.Configure set itself its AppDir ... ?
  • Build : must separate applications / deployement. We must use a interface, where we can pass on the information needed by both parties so we don't really which app are we deploying (for example. we must absolutly remove the need to construct the Merkle tree inside the deployement !)

Localhost-limits

I tried to run the localhost-simulation with 256 collective signers and it stopped without any comment...

Localhost-monitoring

I would like to have a working monitoring for the localhost-simulation, so we can have some quick tests on a local computer.

Userfriendly conode-distribution

Some more ideas for the conode-binaries:

  • fix root-nodes 'connection closed' when a client drops
  • check why root-node closes connections after a wrong signing request
  • add signature-checking of new packets to script

Import proto/sign to lib/sign

Re-think the usefulness of the proto-directory - is it OK to just port it back to the lib-directory? Should it be done in a way that we can import proto/sign or import proto/randhound or import proto/shamir and this way clean up the app/*-directory? This way the proto-directory could be used directly by third-party apps.

Or we just throw away the proto-directory and include it in sign

Check Deterlab

Do some basic testing for deterlab-simulation:

  • Are the servers up?
  • Can each server contact the others?

Implement IPv6 support

Since it is to be expected that IPv4 addresses will become harder and harder to come by in the future, I guess it would make sense to have IPv6 support in the cothority framework to be independent of that "resource". I presume that the realisation of that is not too much of a trouble, as Golang's net package already supports IPv6. There might be even other reasons why we might want to have IPv6 support that I have not thought of yet. Any input is welcome.

Error sign_single.toml localhost

Once again, debug printing is causing an error.
go build && ./deply -platform localhost -debug 3 simulations/sign_single.tomlworks great.
go build && ./deply -platform localhost -debug 1 simulations/sign_single.toml does not. It raises an error as "connection closed".
Tested in current development branch at commit b41200b

Same apply for stamp_single.toml...

Deploy/deterlab: don't clean up before building binaries

The deterlab-platform should stop cleaning up before building the binaries. As this was used to quickly clean up when something was wrong, also add a -clean parameter that only calls the Platform.Clean, without doing Platform.Build

Measurements

Perhaps we should use better measurements than wall-clocking what is happening for each round. What are the possibilities to measure directly in the go-program how much cpu-power we used?

Implementing new baseline

Implement an even simpler baseline where the root asks a number of clients to sign a hash and return the signature. No Shamir/collective signing.

Tests

A lot of tests are failing - we should clean them up.

Clean up only once and not twice

When running the simulation on deterlab, the clean-up code is run twice. It should run only once, as it takes quite some time to ssh to all servers and kill the processes, especially when there is none running

Cleaning up hosts on Deterlab

When running experiments on deterlab, only the used hosts are cleaned up with remaining apps that run. This can create problems if you run first an experiment with 16 hosts, then one with 4 hosts. The 12 remaining hosts aren't cleaned up.

This Quickfix will include the node_list-command from users.deterlab.net to clean up all apps from all started nodes, regardless of how many nodes are used for that run.

It is already included in matplotlib_with_new_stats, but will find its way as a separate pull-request.

Unify configuration-files

Arrange the configuration-files of the different platforms together into one big configuration-file that can also hold more global parameters like:

  • debug - the debug-level
  • monitorPort - where the monitor listens

For the simulation/*toml - files options like discard should also be available.

Graphing of results

Put results of test-runs into graph. Three possibilities:

  1. Matplotlib standalone
  2. plotinum for direct integration into deploy
  3. PGFplots to include the graphs directly into the LaTeX-file

Discard first round in monitoring

Often the first round in an experiment still sets up some structures, so it's often best to discard that first round. I propose per default to discard it, but add an argument to deploy to keep the first round.

Cleanup coconet

A small patch (not including other cleanups) to replace the stamp/sign-related code in coconet.

app/conode round-storage

When app/conode is run, all rounds are stored locally. They should be discarded after:

  • finishing the round
  • more than 100 are stored

Debug Root

It would be nice to have a dbg.PrintRoot or something, so we don't have a to play with the levels to get important informations.

Organise branching

I think we need to organise branching and merging better.

Nicolas found this interesting link which looks nice, IMO:

http://nvie.com/posts/a-successful-git-branching-model/

I propose we implement such a new 'development' branch. Even though we have then to define how a feature branch gets merged into the 'development' branch - which only pushes the problem of when to merge a bit further.

Deploy: Remove first cleanup

When deploy is first started, it calls users -kill to clean up. But the config.toml is not copied yet, so the wrong experiment might be cleaned up.

Either copy over config.toml from the start, or remove the first cleanup before compiling the app.

Change Message to map of bytes

In lib/sign/messagesign.go change all Messages in AnnounceMessage and others to

Message map[string][]byte

so that each round-type can store it's own message.

Problem so far: protobuf doesn't encode this map properly.

English usage of spaces

In English, there is NO space before the ':' (contrary to French), so you write

Launching proxy redirecting to localhost: 4001

Whatsmore the dbg.*-functions that take multiple arguments already enter a space between all arguments, so

dbg.Lvl1("Launching proxy redirecting to ", host )

has one space too much after the to.

deleting "remote"

When deleting "remote" folder on "users.deterlab.net", I got this error:
bash: line 0: cd: remote: No such file or directory
bash: ./users: No such file or directory

Localhost stop before test is finished

In localhost simulation, the Monitor() directly returns if the platform is not deterlab. When monitor returns, the "test" is finished, and a call to platform.Stop() is called. Depending on the situation, it can happens (it has), that apps do not have enough time to finish their test before being killed by the localhost platform.

We should think about that, and this issue may be part of a logging facility restructuration since this is clearly a logging issue...

Bug in Node::actOnCommits

func (sn *Node) actOnCommits(view, Round int) error {
    round := sn.Rounds[Round]
    var err error

    if sn.IsRoot(view) {
        // BUG: when removing the dbg.Lvl5, returns 'invalid elliptic curve'
        dbg.Lvl5("Commit root : Aggregate Public Key :", round.X_hat)

And everybody's wondering why??

Error open test file

From fresh restructure_mains branch, first time I launch :

$ go build && ./deploy -machines 3 -debug 2 simulation/shamir_single.toml
1: (                               main.main:  35) - Deploying to deterlab
1: (                      app.ReadTomlConfig: 153) - Didn't find deter.toml in /Users/gailly/Documents/go/src/github.com/dedis/cothority/deploy
1: (platform.(*Deterlab).LoadAndCheckDeterlabVars: 393) - Couldn't read config-file - asking for default values
Please enter the hostname of deterlab [users.deterlab.net]:
Please enter the login-name on users.deterlab.net []: nikkolas
Please enter the project on deterlab [SAFER]:
Please enter the Experiment on SAFER [Dissent-CS]: DISSENT-CS2
FATA[0037] error opening test file:open test_data/shamir_single.csv: no such file or directory

Change all log to dbg

We're still using log in some parts of lib/sign - this should be changed to dbg calls. But first dbg needs methods for:

  • dbg.Error
  • dbg.Errorf
  • dbg.Warn
  • dbg.Warnf

None of these is panicing or anything else. Just print Error: and the message or Warn: and the message.

Localhost StampClient

Actually, there is no stamping client running on localhost, but only on deterlab.go (these are launched by users.go)

Development-branch

Is there any recommendation about keeping the feature-branches around once they are merged in the development-branch?

Localhost-wait

It doesn't wait for the end of the experiment before quitting to the command line...

localhost - machines & hpn

How should the machines entry and the hpn entry behave in the localhost platform? Personally I would like it to spawn machines * hpn processes. For the moment it spawns machines processes only.

Cothorityd

Implement a simple cothority-daemon for deployment in servers. In a first step, get a simple timestamper-app working that is available through a binary network-interface (only binary messages):

  • configuration-file is static and updated by us
  • tree is static and pre-configured (need to be able to re-create tree based on hostnames and public keys)
  • check on inner workings with regard to collection of Merkle-tree and inclusion-proofs
  • add simple command-line interface that asks for the signature of a file (calculates the hash itself)

Add localhost-simulator

In addition to the deterlab-simulation, add a possibility to run a number of processes on the local host.

Deploy: Check on experiment not active

When running an experiment, deploy/platform/users/users.go checks for all servers using the command node_list on users.deterlab.net. When the experiment is not started yet, node_list returns experiment is not active. This should get users to abort and return an error, which should get deploy to quit.

Move stamplistener to messagessign

For the moment we have two ports open: 2000 for inter-node communication and 2001 for stamp-requests. The stamp-requests (with stamplistener) should also move to lib/sign/messagessign.

A new message-type should be defined that can take a binary which will be stored together with its connection in the Node-structure.

That way we don't have to poke two holes in the firewalls and handle the two-port mess.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.