Most of the configuration has been added. We need to raise a JIRA with sonotype to get the nexus access.

https://central.sonatype.org/pages/ossrh-guide.html

Test memory profiling and update in readme

I want to use mysql to store the information about tessera. I want to kown which version of mysql I need to use and What the table structure I need to define ? Thank you.

The current configuration has no clear means of separating node to node communication (party info) and the endpoints used for privacy management that quorum uses.

To have a cluster of nodes against each quorum node to resume transaction processing even if one node in the cluster is down. All nodes in cluster will share all private/public key pairs associated with that quorum node. The connection is to listen over HTTP or gRPC with robust authentication with a load balancing layer managing the cluster

Benchmarking & profiling. See how to improve

Tessera should be readily configurable to any DB supporting jdbc driver. Test with various DBs and provide config info

This is mainly used for backups / regulatory node etc...

Organisation change

Go through and identify all items in Constellation config files that is not in Tessera config and add support to all through migration script/toolset.

not in tessera config that is in constellation

Looks like this on my mac:

 ✘ fixanoid@Felixs-MacBook-Pro  ~/Projects/tessera   master  tessera help  
usage: tessera -configfile <PATH> [-keygen <PATH>] [-pidfile <PATH>]
    --alwaysSendTo.keyBytes <BYTE[]...>                         Override
                                                                option for
                                                                alwaysSend
                                                                To.keyByte
                                                                s , type:
                                                                byte[]
 -configfile <PATH>                                             Path to
                                                                node
                                                                configurat
                                                                ion file
    --jdbc.password <STRING>                                    Override
                                                                option for
                                                                jdbc.passw
                                                                ord ,
                                                                type:
                                                                String

All of the param explanations are like this.

We lost a lot of benefit of having a structured xml means for providing configuration before we broke it swapping to a purely json and worse supporting toml file .. now we've created several strands of tech debt reactively just implementing things without sufficient challenging. And therefore risk to the success of the overall program.

The TOML config is particularly problematic as it attempt to align the private key paths, public key paths and the contents of the passwords file to create a key config. Worse still the private key contents are intact key generation options structured as son fragments.

all classes in tessera core should have the base package com.quorum.tessera.core

this rule of having a module reference in the package name should be applied to all of the modules.

com.quorum.tessera.{module ref}

having a distinct namespace will mean tessera can be more easily packaged for java 9+ module-info descriptors, as modules cannot have package name clashes.

I have installed mysql on local machine
I have downloaded the jar(version 8.0.12) about mysql jdbc and saved at /home/vagrant/tessera/mysql.jar
I have run the ddl(https://github.com/jpmorganchase/tessera/blob/master/ddls/mysql-ddl.sql) of mysql

I modify the tessera-init.sh as follwoing and change the jdbc url to mysql

#!/usr/bin/env bash
echo "[*] Initialising Tessera configuration"

currentDir=$(pwd)
for i in {1..7}
do
DDIR="qdata/c$i"
mkdir -p ${DDIR}
mkdir -p qdata/logs
cp "keys/tm$i.pub" "${DDIR}/tm.pub"
cp "keys/tm$i.key" "${DDIR}/tm.key"
rm -f "${DDIR}/tm.ipc"
#change tls to "strict" to enable it (don't forget to also change http -> https)
cat < ${DDIR}/tessera-config${i}.json
{
"useWhiteList": false,
"jdbc": {
"username": "root",
"password": "123456",
"url": "jdbc:mysql//127.0.0.1:3306/quorum?useSSL=false"
},
"server": {
"port": 900${i},
"hostName": "http://localhost",
"sslConfig": {
"tls": "OFF",
"generateKeyStoreIfNotExisted": true,
"serverKeyStore": "${currentDir}/qdata/c${i}/server${i}-keystore",
"serverKeyStorePassword": "quorum",
"serverTrustStore": "${currentDir}/qdata/c${i}/server-truststore",
"serverTrustStorePassword": "quorum",
"serverTrustMode": "TOFU",
"knownClientsFile": "${currentDir}/qdata/c${i}/knownClients",
"clientKeyStore": "${currentDir}/qdata/c${i}/client${i}-keystore",
"clientKeyStorePassword": "quorum",
"clientTrustStore": "${currentDir}/qdata/c${i}/client-truststore",
"clientTrustStorePassword": "quorum",
"clientTrustMode": "TOFU",
"knownServersFile": "${currentDir}/qdata/c${i}/knownServers"
}
},
"peer": [
{
"url": "http://localhost:9001"
},
{
"url": "http://localhost:9002"
},
{
"url": "http://localhost:9003"
},
{
"url": "http://localhost:9004"
},
{
"url": "http://localhost:9005"
},
{
"url": "http://localhost:9006"
},
{
"url": "http://localhost:9007"
}
],
"keys": {
"passwords": [],
"keyData": [
{
"config": $(cat ${currentDir}/qdata/c${i}/tm.key),
"publicKey": "$(cat ${currentDir}/qdata/c${i}/tm.pub)"
}
]
},
"alwaysSendTo": [],
"unixSocketFile": "${currentDir}/qdata/c${i}/tm.ipc"
}
EOF
done

com.github.nexus.config.adapters.KeyDataAdapter needs to be interpreted with the create pairs code in KeyManager located in the nexus-app module. The KeyManger code that does this needs to be in the config module and all of this needs to happen in the config module.

    @Override
    public KeyData unmarshal(KeyData keyData) throws Exception {

        if (keyData.hasKeys()) {
            return keyData;
        }

        //TODO: Generate key pairs from and create populated keyData
        KeyDataConfig keyDataConfig = KeyDataConfigStore.INSTANCE.pop();

        return new KeyData(keyDataConfig, null, null);
    }

The private and public keys need generating from the KeyDataConfig object and storing on the KeyData

Strategically we will have key management tools integrated onto Tessera

Hello. I was trying install and compile tessera.

1. $ java -version
   java version "1.8.0_181"
   Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
   Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

2. $ mvn --version
   Apache Maven 3.3.9
   Maven home: /usr/share/maven
   Java version: 1.8.0_181, vendor: Oracle Corporation
   Java home: /usr/lib/jvm/java-8-oracle/jre
   Default locale: en_US, platform encoding: UTF-8
   OS name: "linux", version: "4.15.0-1022-azure", arch: "amd64", family: "unix"

3. Apache Tomcat/9.0.11

I cloned this repo, switch folder and tried run mvn install but got error:

I used basic config.json

What did I do wrong?

I don't know why, but when i'm executing unit tests in the cli module i end up seeing the following

System.out.println("Enter a relative or absolute path (without extension) to save the keys to");
System.out.println("or leave blank to not save to separate file:");
final String path = new Scanner(filenameStream).nextLine();

if the fileNameStream doesn't have a next line it just sits. It looks like the object should throw an exception in this case or something a CLI can use to inform the user that something is afoot.

In the README :
Change mvn install -Pkalium to mvn install -P kalium.

• Bonus: It would be nice to have a 'or' between jnacl and kalium for better understanding.
  Thanks for your work !

Rather than masking configuration output to avoid printing privateKey values and/or passwords we have jaxb adapters that null values while they are being marshalled. This has broken override cli args and makes the entire config loading unpredictable and unmaintainable.

1. Add a masking marshaller to deal with these case
2. Check that the new function is always used when logging or/and printing to console

Tests that create files and remove the writable attribute so an exception is thrown for the test don't work as the root user (on Linux) because the root user always has write access, regardless of permissions.

Tests that use File#setWritable(false) are the ones that are failing. An alternative way to ensure the exception gets thrown should be used that works with root/admin users.

the current process used to create 0.5.2

1. Create and switch to writable branch from master "release from-master"

mvn clean release:prepare -B

Note: The clean isn't required but was there when I did it.

mvn clean release:perform -B

After the tag was created on git hub there's a last step to create release from tag.

Issues/manual steps

1. Creating the branch from master as can't write to master without a PR
2. needed to run test suite multiple times, that might add to the 10 minute build time for the release:perform
3. Release publish step (could be fine for now)

There's a legacy file system data format. Our present code tries to leverage jdbc as much as possible.

Assuming that is appetite for directory system datastore support the choice is find a jdbc driver that allows us to do this which will give us the benefit of less code changes or add specialised support by way of a FileSystemDAO or something. This will mean more code and having a less homogenous code base.

After alias tessera=java\ -jar\ ~/Projects/tessera/tessera-app/target/tessera-app-1.0-SNAPSHOT-app.jar and launching tessera prints errors. Should default to help.

The encryption-api module defines enclave as the service that core uses for encryption operations and key access. Private keys should never be exposed outside of its system boundary.

One of the proposed deployment options is that enclave can be co-deployed in the same location as private keys and expose its services to tessera which is potentially deployed in a different location.

Task 1. Provide optional enclave api/transport options to communicate with tessera core.
Task 2. Provide optional configuration that instructs tessera core how to access enclave securely.

The GRPC acceptance tests do some basic checks to ensure a GRPC service is running, but a full set of tests for each endpoint should be made to ensure the calls act as expected, similar to the REST acceptance tests.

I have a conceptual question regarding Tessera/Constellation and private key encryption/decryption.
https://github.com/jpmorganchase/tessera/wiki/Configuration#using-existing-keys

I would like to encrypt a private key using your tools. It looks like that the private key derived from the password is directly used as the Tessera private key. Am I correct with this observation?

If so, this means that there is a strong dependency between the password, the private key and the public key. If I want to change the password for what ever reason (e.g. compromised password), this means that I can't "keep" my public key, since password => private key => public key.

Would it be feasible to add another "encryption mode" where the derived key isn't the actual private key, rather an intermediary private key to encrypt/decrypt the actual private key (like geth does it)? Are there any alternatives you would recommend instead (using a 3rd party solution) for encrypting the key with a password that can be rotated?

This should include node discovery ON/OFF and addition of peer nodes.

com.quorum.tessera.config.keys.KeyGeneratorFactory needs use the service loader or other mechanism so the Cli and other modules that use it aren't forced to use the implementation just to run some unit tests that only need to prove that there is an interaction with the api.

DDL Added:

tessera/tessera-app/src/main/resources/tesseraDDL.sql

CREATE TABLE ENCRYPTED_TRANSACTION (ID BIGINT NOT NULL, ENCODED_PAYLOAD LONGVARBINARY NOT NULL, HASH LONGVARBINARY NOT NULL UNIQUE, PRIMARY KEY (ID))
CREATE SEQUENCE ENC_TX_SEQ INCREMENT BY 50 START WITH 50

The tessera examples need maintaining and executing as part of the main project. Or as a minimum have the ability to read the current (or user defined) tessera version and run from that.

At the moment tessera-core reuses the api objects as its domain model rather than defining its own domain model.

Task 1. Create domain model in core and redefine transaction manager to use them
Task 2. Move com.quorum.tessera.api.model in tessera core into jax rs module. Provide convertors to copy between domain and api objects.

Implement a common set of interfaces to ensure that the read methods are defined on api and domain.

When private and public keys are generated. using -keygen an additional -output or needs defining.

The default -output should be to sys.out if not defined.. if a path is define the config should be marshalled to that and exit wit running.

While code coverage is good, some of the more complex override test cases aren't in the suite.

When a user elects to generate a key from the tessera cli s/he apparently can provide an empty file name. Normal usage is :

tessera -keygen -filename somefile,someotherfile

apparently its desirable to also support

tessera -keygen -filename

In this case we've now gone to the effort of defining the filename option and then not provided any values for it, the reasonable exception would be that the user is then informed that they haven't defined the file name(s) s/he wishes to use to generate the keys.

Once we clarify that having no args for -filename is incorrect we need to remove support for it.

The advertised URL may be different the the port the web server is bound on, and the configuration should be separated.

Methodology: Create tessera alias and then cd to tressera/config then run using sample files.

1. Misleading message when public key path is invalid.

tessera -configfile src/test/resources/keytests/pubPrivUsingPathsUnlocked_missingPublicKey.json --unixSocketFile /tmp/foo.ipc
Config validation issue: keys.keyData KeyData 1 : Public key path BOGUS.pub does not exist

FIXED.

2. Misleading message when private key path is invalid

tessera -configfile src/test/resources/keytests/pubPrivUsingPathsUnlocked_missingPrivateKey.json --unixSocketFile /tmp/foo.ipc
Config validation issue: keys.keyData KeyData 1 : Private key path BOGUS.key does not exist

FIXED.

3. pubPrivUsingPathsLocked case locks and provides on output
   tessera -configfile src/test/resources/keytests/pubPrivUsingPathsLocked.json --unixSocketFile /tmp/foo.ipc
   Expectation is to for cli to return some feedback, but this happy case should result in success

4. passwordsMissing.json

tessera -configfile src/test/resources/keytests/passwordsMissing.json --unixSocketFile /tmp/foo.ipc
Config validation issue: keys.keyData[0].config A locked key was provided without a password.
 Please ensure the same number of passwords are provided as there are keys and remember to include empty passwords for unlocked keys

FIXED

5. Wrong passwords

tessera -configfile src/test/resources/keytests/passwordsWrong.json --unixSocketFile /tmp/foo.ipc
Could not decrypt the private key with the provided password, please double check the passwords provided
Config validation issue: keys.keyData[0].privateKey Could not decrypt the private key with the provided password, please double check the passwords provided

FIXED

6. Invalid Unix domain socket file path isn't validated and ends up with big stacktrace on server startup

tessera -configfile src/test/resources/keytests/pubPrivUsingPathsUnlocked.json
Config validation issue: unixSocketFile Unable to create file /Users/peter/IdeaProjects/quorum-ex/examples/7nodes/qdata/c1/tm.ipc

FIXED

At the moment a pid file system property is used to define an optional pid file. Usage usage should be

nexus -pidfile /somepath/my.pid ..

rather than java -Dnexus.pid.file=/somepath/my.pid

The makes the nexus alias more portable, using the system property means the pid will be used for all invocations.