dedis / cothority Goto Github PK

In localhost simulation, the Monitor() directly returns if the platform is not deterlab. When monitor returns, the "test" is finished, and a call to platform.Stop() is called. Depending on the situation, it can happens (it has), that apps do not have enough time to finish their test before being killed by the localhost platform.

We should think about that, and this issue may be part of a logging facility restructuration since this is clearly a logging issue...

In English, there is NO space before the ':' (contrary to French), so you write

Launching proxy redirecting to localhost: 4001

Whatsmore the dbg.*-functions that take multiple arguments already enter a space between all arguments, so

dbg.Lvl1("Launching proxy redirecting to ", host )

has one space too much after the to.

Is there any recommendation about keeping the feature-branches around once they are merged in the development-branch?

Implement an even simpler baseline where the root asks a number of clients to sign a hash and return the signature. No Shamir/collective signing.

When running an experiment, deploy/platform/users/users.go checks for all servers using the command node_list on users.deterlab.net. When the experiment is not started yet, node_list returns experiment is not active. This should get users to abort and return an error, which should get deploy to quit.

Do some basic testing for deterlab-simulation:

• Are the servers up?
• Can each server contact the others?

Since it is to be expected that IPv4 addresses will become harder and harder to come by in the future, I guess it would make sense to have IPv6 support in the cothority framework to be independent of that "resource". I presume that the realisation of that is not too much of a trouble, as Golang's net package already supports IPv6. There might be even other reasons why we might want to have IPv6 support that I have not thought of yet. Any input is welcome.

Once again, debug printing is causing an error.
go build && ./deply -platform localhost -debug 3 simulations/sign_single.tomlworks great.
go build && ./deply -platform localhost -debug 1 simulations/sign_single.toml does not. It raises an error as "connection closed".
Tested in current development branch at commit b41200b

Same apply for stamp_single.toml...

Often the first round in an experiment still sets up some structures, so it's often best to discard that first round. I propose per default to discard it, but add an argument to deploy to keep the first round.

Some more ideas for the conode-binaries:

• fix root-nodes 'connection closed' when a client drops
• check why root-node closes connections after a wrong signing request
• add signature-checking of new packets to script

A small patch (not including other cleanups) to replace the stamp/sign-related code in coconet.

There's an issue with mutli-line code formatting in the conode README.

How should the machines entry and the hpn entry behave in the localhost platform? Personally I would like it to spawn machines * hpn processes. For the moment it spawns machines processes only.

When deleting "remote" folder on "users.deterlab.net", I got this error:
bash: line 0: cd: remote: No such file or directory
bash: ./users: No such file or directory

Arrange the configuration-files of the different platforms together into one big configuration-file that can also hold more global parameters like:

• debug - the debug-level
• monitorPort - where the monitor listens

For the simulation/*toml - files options like discard should also be available.

When deploy is first started, it calls users -kill to clean up. But the config.toml is not copied yet, so the wrong experiment might be cleaned up.

Either copy over config.toml from the start, or remove the first cleanup before compiling the app.

The default port should be 10001 instead of 4001, so that it doesn't interfere with the app-ports.

When running the simulation on deterlab, the clean-up code is run twice. It should run only once, as it takes quite some time to ssh to all servers and kill the processes, especially when there is none running

Perhaps we should use better measurements than wall-clocking what is happening for each round. What are the possibilities to measure directly in the go-program how much cpu-power we used?

In lib/sign/messagesign.go change all Messages in AnnounceMessage and others to

Message map[string][]byte

so that each round-type can store it's own message.

Problem so far: protobuf doesn't encode this map properly.

I would like to have a working monitoring for the localhost-simulation, so we can have some quick tests on a local computer.

Add a config-option for the baseline to take away the checking of the signatures

At the end of a measurement the ssh of the proxy should be killed.

Actually, there is no stamping client running on localhost, but only on deterlab.go (these are launched by users.go)

Implement a simple cothority-daemon for deployment in servers. In a first step, get a simple timestamper-app working that is available through a binary network-interface (only binary messages):

• configuration-file is static and updated by us
• tree is static and pre-configured (need to be able to re-create tree based on hostnames and public keys)
• check on inner workings with regard to collection of Merkle-tree and inclusion-proofs
• add simple command-line interface that asks for the signature of a file (calculates the hash itself)

I think we need to organise branching and merging better.

Nicolas found this interesting link which looks nice, IMO:

http://nvie.com/posts/a-successful-git-branching-model/

I propose we implement such a new 'development' branch. Even though we have then to define how a feature branch gets merged into the 'development' branch - which only pushes the problem of when to merge a bit further.

A lot of tests are failing - we should clean them up.

In monitor remove the extraFields and additionals measures so that it displays every fields that it gets and every measures it receives.

We're still using log in some parts of lib/sign - this should be changed to dbg calls. But first dbg needs methods for:

• dbg.Error
• dbg.Errorf
• dbg.Warn
• dbg.Warnf

None of these is panicing or anything else. Just print Error: and the message or Warn: and the message.

It doesn't wait for the end of the experiment before quitting to the command line...

Can we use somewhat more concise directory/package names, when they'll still be clear but save a bit of typing and screen space? Suggestions:

application -> app
protocols -> proto
helpers -> lib?

Verify that Conode uses the 8x-Edwards suite and do a new binary and tell everybody to redo it's key.

Under some circumstances, the protobuf just returns an empty slice. For example when Encoding a AnnouncementMessage with an uninitialised Message.

Because some of the runs take a lot more time in some special cases, we'd like to measure only the fastest measurements. So instead of only dropping the first measurement, drop the 'n' slowest measurements, where 'n' is a variable available on the command-line or in the configuration-script.

Things noted in deploy that may need to be changed:

• Deterlab.Configure set itself its AppDir ... ?
• Build : must separate applications / deployement. We must use a interface, where we can pass on the information needed by both parties so we don't really which app are we deploying (for example. we must absolutly remove the need to construct the Merkle tree inside the deployement !)

I tried to run the localhost-simulation with 256 collective signers and it stopped without any comment...

Put results of test-runs into graph. Three possibilities:

1. Matplotlib standalone
2. plotinum for direct integration into deploy
3. PGFplots to include the graphs directly into the LaTeX-file

When app/conode is run, all rounds are stored locally. They should be discarded after:

• finishing the round
• more than 100 are stored

When running experiments on deterlab, only the used hosts are cleaned up with remaining apps that run. This can create problems if you run first an experiment with 16 hosts, then one with 4 hosts. The 12 remaining hosts aren't cleaned up.

This Quickfix will include the node_list-command from users.deterlab.net to clean up all apps from all started nodes, regardless of how many nodes are used for that run.

It is already included in matplotlib_with_new_stats, but will find its way as a separate pull-request.

When the ssh-key is protected by a password, deploy can't connect to the users.deterlab.net-server without having to ask for the password all the time.

Use ssh-agent or any other convenient tool to bypass the password-request.

From fresh restructure_mains branch, first time I launch :

$ go build && ./deploy -machines 3 -debug 2 simulation/shamir_single.toml
1: (                               main.main:  35) - Deploying to deterlab
1: (                      app.ReadTomlConfig: 153) - Didn't find deter.toml in /Users/gailly/Documents/go/src/github.com/dedis/cothority/deploy
1: (platform.(*Deterlab).LoadAndCheckDeterlabVars: 393) - Couldn't read config-file - asking for default values
Please enter the hostname of deterlab [users.deterlab.net]:
Please enter the login-name on users.deterlab.net []: nikkolas
Please enter the project on deterlab [SAFER]:
Please enter the Experiment on SAFER [Dissent-CS]: DISSENT-CS2
FATA[0037] error opening test file:open test_data/shamir_single.csv: no such file or directory

The deterlab-platform should stop cleaning up before building the binaries. As this was used to quickly clean up when something was wrong, also add a -clean parameter that only calls the Platform.Clean, without doing Platform.Build

It would be nice to have a dbg.PrintRoot or something, so we don't have a to play with the levels to get important informations.

Re-think the usefulness of the proto-directory - is it OK to just port it back to the lib-directory? Should it be done in a way that we can import proto/sign or import proto/randhound or import proto/shamir and this way clean up the app/*-directory? This way the proto-directory could be used directly by third-party apps.

Or we just throw away the proto-directory and include it in sign

func (sn *Node) actOnCommits(view, Round int) error {
    round := sn.Rounds[Round]
    var err error

    if sn.IsRoot(view) {
        // BUG: when removing the dbg.Lvl5, returns 'invalid elliptic curve'
        dbg.Lvl5("Commit root : Aggregate Public Key :", round.X_hat)

And everybody's wondering why??

For the moment we have two ports open: 2000 for inter-node communication and 2001 for stamp-requests. The stamp-requests (with stamplistener) should also move to lib/sign/messagessign.

A new message-type should be defined that can take a binary which will be stored together with its connection in the Node-structure.

That way we don't have to poke two holes in the firewalls and handle the two-port mess.

We start using slack now...

In addition to the deterlab-simulation, add a possibility to run a number of processes on the local host.

There seems to be a one-off bug at https://github.com/DeDiS/cothority/blob/cosi_linus/app/stamp/server.go#L267. As a consequence, simulations run one round less compared to the "Rounds" value specified in the configuration .toml files.