https://github.com/cisco/go-mls/blob/master/state.go#L680

The next.TreePriv.Decap and next.Tree.Merge runs independently for the public key and private key part of the tree.
But the secrets and public key in DirectPath is not verified against each other.

Hi there.
I have tried to build your project and run your tests, using gollvm .

WORK=/tmp/go-build745669351
mkdir -p $WORK/b001/
mkdir -p $WORK/b030/
mkdir -p $WORK/b055/
mkdir -p $WORK/b016/
mkdir -p $WORK/b026/
mkdir -p $WORK/b058/
mkdir -p $WORK/b060/
mkdir -p $WORK/b067/
mkdir -p $WORK/b031/
mkdir -p $WORK/b063/
mkdir -p $WORK/b062/
mkdir -p $WORK/b059/
mkdir -p $WORK/b066/
mkdir -p $WORK/b068/
mkdir -p $WORK/b056/
mkdir -p $WORK/b022/
mkdir -p $WORK/b069/
mkdir -p $WORK/b065/
mkdir -p $WORK/b071/
mkdir -p $WORK/b053/
mkdir -p $WORK/b021/
mkdir -p $WORK/b008/
cat >$WORK/b001/importcfg.link << 'EOF' # internal
packagefile github.com/cisco/go-mls.test=/home/oceanfish81/.cache/go-build/a4/a49f5f655fee45e9061e3ce99eecc6ab1779202b341e38b10c505199e180b640-d
packagefile github.com/cisco/go-mls=/home/oceanfish81/.cache/go-build/46/46c1fdb32399b31645d9d92a8eef6873f7b0632ce5dfeebc03a50d04090e8314-d
packagefile github.com/cisco/go-tls-syntax=/home/oceanfish81/.cache/go-build/38/386780eec56678e3f9bd060f4996463284458962c4823105d100e3fe886836bb-d
packagefile github.com/stretchr/testify/require=/home/oceanfish81/.cache/go-build/22/22658188308d3dfb26a8ebd48e9db7a23d4423d7e8edf7476b4eb4794e7eab66-d
packagefile github.com/cisco/go-hpke=/home/oceanfish81/.cache/go-build/30/30ff53905c8069d2b504f95127901daa60dae7ed14040abd16d504e8db834eb7-d
packagefile golang.org/x/crypto/chacha20poly1305=/home/oceanfish81/.cache/go-build/b2/b20ccf81e8c8edcd58b3380ac422e7ffeac602314a881e846c88ebb3a06ae7e4-d
packagefile golang.org/x/crypto/ed25519=/home/oceanfish81/.cache/go-build/41/410c93aff478a856645491b577167425a11deafe127f3221f1f5b1f7631c512d-d
packagefile github.com/stretchr/testify/assert=/home/oceanfish81/.cache/go-build/85/85c370f4495b07c435cc32d2b1e90b983d5d00e4c6cd73f530b0013713665a0a-d
packagefile git.schwanenlied.me/yawning/x448.git=/home/oceanfish81/.cache/go-build/d5/d584cd6fe3934f8d21a76a02eeb40dc0bde5a05bc0c2770758fa9f03d9f2a38f-d
packagefile github.com/cloudflare/circl/dh/sidh=/home/oceanfish81/.cache/go-build/8c/8c215d5de55ca9a35bded57b0f3559f39422cdde355f43277bf3b783f02d02ff-d
packagefile golang.org/x/crypto/curve25519=/home/oceanfish81/.cache/go-build/4c/4cc0716b72970e43141485dead61490c7faded2a3947b9ccbbbea8dac01410c3-d
packagefile golang.org/x/crypto/chacha20=/home/oceanfish81/.cache/go-build/0f/0f4772b0813cdb3e8be465dfe2396327bd7252b3da835d16703c0344d4413dd9-d
packagefile golang.org/x/crypto/internal/subtle=/home/oceanfish81/.cache/go-build/a6/a61b84435ebdfa9a376cbfec112030903f393826fb5d14c16f7589f3bca4af48-d
packagefile golang.org/x/crypto/poly1305=/home/oceanfish81/.cache/go-build/7a/7aa798315e9477bb299ee1838a13c455be875d986dca716bab88de8b684d7b53-d
packagefile github.com/davecgh/go-spew/spew=/home/oceanfish81/.cache/go-build/1b/1b7d599e5c381d0855655ad97985e6b119755fbae80e55bd37bd8f0581a4354f-d
packagefile github.com/pmezard/go-difflib/difflib=/home/oceanfish81/.cache/go-build/70/70c764160d4d4bafe3b50d510deb1c71c1f53f3776e03397dc5370c0f3b581b0-d
packagefile gopkg.in/yaml.v3=/home/oceanfish81/.cache/go-build/84/8479df47a78a204c41f73d6717804662586a7a33cfdace4666d0040ff028dbf3-d
packagefile github.com/cloudflare/circl/dh/sidh/internal/common=/home/oceanfish81/.cache/go-build/f3/f3ecaeb380bd7dafcdfd06897e0994b61f82fb12915c634d71f93192d71d752e-d
packagefile github.com/cloudflare/circl/dh/sidh/internal/p503=/home/oceanfish81/.cache/go-build/0b/0b021a0b020e4c9684aa52280f16643f5a264edfebf2a9a2f3bbd9f4d6c02c74-d
packagefile github.com/cloudflare/circl/dh/sidh/internal/p751=/home/oceanfish81/.cache/go-build/e2/e27f1b8c8790932db558e18298d34976cb1dc505c9bcc94faf060e2d121f3834-d
packagefile github.com/cloudflare/circl/dh/sidh/internal/shake=/home/oceanfish81/.cache/go-build/37/377d4565b639d3db3ef937c79b71f84eadac3e2d308a824331acb78c1ace059c-d
packagefile golang.org/x/sys/cpu=/home/oceanfish81/.cache/go-build/29/29fd5ddc449865d135ddade00e4118afda8b921530193b30e0480ad6fa2bc12f-d
EOF
cd .
/usr/local/bin/llvm-goc -o $WORK/b001/go-mls.test "-Wl,-(" -m64 -Wl,--whole-archive /home/oceanfish81/.cache/go-build/a4/a49f5f655fee45e9061e3ce99eecc6ab1779202b341e38b10c505199e180b640-d /home/oceanfish81/.cache/go-build/46/46c1fdb32399b31645d9d92a8eef6873f7b0632ce5dfeebc03a50d04090e8314-d /home/oceanfish81/.cache/go-build/38/386780eec56678e3f9bd060f4996463284458962c4823105d100e3fe886836bb-d /home/oceanfish81/.cache/go-build/22/22658188308d3dfb26a8ebd48e9db7a23d4423d7e8edf7476b4eb4794e7eab66-d /home/oceanfish81/.cache/go-build/30/30ff53905c8069d2b504f95127901daa60dae7ed14040abd16d504e8db834eb7-d /home/oceanfish81/.cache/go-build/b2/b20ccf81e8c8edcd58b3380ac422e7ffeac602314a881e846c88ebb3a06ae7e4-d /home/oceanfish81/.cache/go-build/41/410c93aff478a856645491b577167425a11deafe127f3221f1f5b1f7631c512d-d /home/oceanfish81/.cache/go-build/85/85c370f4495b07c435cc32d2b1e90b983d5d00e4c6cd73f530b0013713665a0a-d /home/oceanfish81/.cache/go-build/d5/d584cd6fe3934f8d21a76a02eeb40dc0bde5a05bc0c2770758fa9f03d9f2a38f-d /home/oceanfish81/.cache/go-build/8c/8c215d5de55ca9a35bded57b0f3559f39422cdde355f43277bf3b783f02d02ff-d /home/oceanfish81/.cache/go-build/4c/4cc0716b72970e43141485dead61490c7faded2a3947b9ccbbbea8dac01410c3-d /home/oceanfish81/.cache/go-build/0f/0f4772b0813cdb3e8be465dfe2396327bd7252b3da835d16703c0344d4413dd9-d /home/oceanfish81/.cache/go-build/a6/a61b84435ebdfa9a376cbfec112030903f393826fb5d14c16f7589f3bca4af48-d /home/oceanfish81/.cache/go-build/7a/7aa798315e9477bb299ee1838a13c455be875d986dca716bab88de8b684d7b53-d /home/oceanfish81/.cache/go-build/1b/1b7d599e5c381d0855655ad97985e6b119755fbae80e55bd37bd8f0581a4354f-d /home/oceanfish81/.cache/go-build/70/70c764160d4d4bafe3b50d510deb1c71c1f53f3776e03397dc5370c0f3b581b0-d /home/oceanfish81/.cache/go-build/84/8479df47a78a204c41f73d6717804662586a7a33cfdace4666d0040ff028dbf3-d /home/oceanfish81/.cache/go-build/f3/f3ecaeb380bd7dafcdfd06897e0994b61f82fb12915c634d71f93192d71d752e-d /home/oceanfish81/.cache/go-build/0b/0b021a0b020e4c9684aa52280f16643f5a264edfebf2a9a2f3bbd9f4d6c02c74-d /home/oceanfish81/.cache/go-build/e2/e27f1b8c8790932db558e18298d34976cb1dc505c9bcc94faf060e2d121f3834-d /home/oceanfish81/.cache/go-build/37/377d4565b639d3db3ef937c79b71f84eadac3e2d308a824331acb78c1ace059c-d /home/oceanfish81/.cache/go-build/29/29fd5ddc449865d135ddade00e4118afda8b921530193b30e0480ad6fa2bc12f-d -Wl,--no-whole-archive "-Wl,-)" -Wl,--build-id=0x327069746b58372d77733054456c7038745a68662f324d387263562d7a5865594e39595a327543416a2f506a3035454d6f54394d304f454d6a62766f72422f327069746b58372d77733054456c7038745a6866
cat >$WORK/b008/vet.cfg << 'EOF' # internal
{
"ID": "github.com/cisco/go-mls",
"Compiler": "gccgo",
"Dir": "/home/oceanfish81/go-mls",
"ImportPath": "github.com/cisco/go-mls",
"GoFiles": [
"/home/oceanfish81/go-mls/common.go",
"/home/oceanfish81/go-mls/credential.go",
"/home/oceanfish81/go-mls/crypto.go",
"/home/oceanfish81/go-mls/extensions.go",
"/home/oceanfish81/go-mls/key-schedule.go",
"/home/oceanfish81/go-mls/messages.go",
"/home/oceanfish81/go-mls/state.go",
"/home/oceanfish81/go-mls/tree-math.go",
"/home/oceanfish81/go-mls/treekem.go",
"/home/oceanfish81/go-mls/common_test.go",
"/home/oceanfish81/go-mls/credential_test.go",
"/home/oceanfish81/go-mls/crypto_test.go",
"/home/oceanfish81/go-mls/extensions_test.go",
"/home/oceanfish81/go-mls/key-schedule_test.go",
"/home/oceanfish81/go-mls/messages_test.go",
"/home/oceanfish81/go-mls/state_test.go",
"/home/oceanfish81/go-mls/test-vectors_test.go",
"/home/oceanfish81/go-mls/tree-math_test.go",
"/home/oceanfish81/go-mls/treekem_test.go"
],
"NonGoFiles": [],
"ImportMap": {
"bytes": "bytes",
"crypto": "crypto",
"crypto/aes": "crypto/aes",
"crypto/cipher": "crypto/cipher",
"crypto/ecdsa": "crypto/ecdsa",
"crypto/ed25519": "crypto/ed25519",
"crypto/elliptic": "crypto/elliptic",
"crypto/hmac": "crypto/hmac",
"crypto/rand": "crypto/rand",
"crypto/sha256": "crypto/sha256",
"crypto/sha512": "crypto/sha512",
"crypto/x509": "crypto/x509",
"encoding/asn1": "encoding/asn1",
"encoding/hex": "encoding/hex",
"fmt": "fmt",
"github.com/cisco/go-hpke": "github.com/cisco/go-hpke",
"github.com/cisco/go-tls-syntax": "github.com/cisco/go-tls-syntax",
"github.com/stretchr/testify/require": "github.com/stretchr/testify/require",
"golang.org/x/crypto/chacha20poly1305": "golang.org/x/crypto/chacha20poly1305",
"golang.org/x/crypto/ed25519": "golang.org/x/crypto/ed25519",
"hash": "hash",
"io/ioutil": "io/ioutil",
"math/big": "math/big",
"math/rand": "math/rand",
"os": "os",
"path/filepath": "path/filepath",
"reflect": "reflect",
"testing": "testing",
"time": "time"
},
"PackageFile": {
"github.com/cisco/go-hpke": "/home/oceanfish81/.cache/go-build/30/30ff53905c8069d2b504f95127901daa60dae7ed14040abd16d504e8db834eb7-d",
"github.com/cisco/go-tls-syntax": "/home/oceanfish81/.cache/go-build/38/386780eec56678e3f9bd060f4996463284458962c4823105d100e3fe886836bb-d",
"github.com/stretchr/testify/require": "/home/oceanfish81/.cache/go-build/22/22658188308d3dfb26a8ebd48e9db7a23d4423d7e8edf7476b4eb4794e7eab66-d",
"golang.org/x/crypto/chacha20poly1305": "/home/oceanfish81/.cache/go-build/b2/b20ccf81e8c8edcd58b3380ac422e7ffeac602314a881e846c88ebb3a06ae7e4-d",
"golang.org/x/crypto/ed25519": "/home/oceanfish81/.cache/go-build/41/410c93aff478a856645491b577167425a11deafe127f3221f1f5b1f7631c512d-d"
},
"Standard": {
"bytes": true,
"crypto": true,
"crypto/aes": true,
"crypto/cipher": true,
"crypto/ecdsa": true,
"crypto/ed25519": true,
"crypto/elliptic": true,
"crypto/hmac": true,
"crypto/rand": true,
"crypto/sha256": true,
"crypto/sha512": true,
"crypto/x509": true,
"encoding/asn1": true,
"encoding/hex": true,
"fmt": true,
"hash": true,
"io/ioutil": true,
"math/big": true,
"math/rand": true,
"os": true,
"path/filepath": true,
"reflect": true,
"testing": true,
"time": true
},
"PackageVetx": {
"github.com/cisco/go-hpke": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/cisco/go-tls-syntax": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"github.com/stretchr/testify/require": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"golang.org/x/crypto/chacha20poly1305": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d",
"golang.org/x/crypto/ed25519": "/home/oceanfish81/.cache/go-build/e3/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855-d"
},
"VetxOnly": false,
"VetxOutput": "$WORK/b008/vet.out",
"SucceedOnTypecheckFailure": false
}
EOF
cd /home/oceanfish81/go-mls
TERM='dumb' GCCGO='/usr/local/bin/llvm-goc' /usr/local/tools/vet -atomic -bool -buildtags -errorsas -ifaceassert -nilfunc -printf -stringintconv $WORK/b008/vet.cfg
$WORK/b001/go-mls.test -test.timeout=10m0s -test.v=true
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4cfb50]

goroutine 1 [running]:
gccgoXgetbv
./vendor/golang.org/x/sys/cpu/cpu_gccgo.c:40
cpu.xgetbv
/home/oceanfish81/go-mls/vendor/golang.org/x/sys/cpu/cpu_gccgo.go:24
golang.x2eorg..z2fx..z2fsys..z2fcpu.golang.x2eorg..z2fx..z2fsys..z2fcpu..init0
/home/oceanfish81/go-mls/vendor/golang.org/x/sys/cpu/cpu_x86.go:37
golang.x2eorg..z2fx..z2fsys..z2fcpu..import
/home/oceanfish81/go-mls/vendor/golang.org/x/sys/cpu/cpu_x86.go:11
exit status 2
FAIL github.com/cisco/go-mls 0.453s
rm -r $WORK/b001/

Here is my environment:

$ go version && go env
go version go1.15rc2 gollvm LLVM 12.0.0git linux/amd64
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/oceanfish81/.cache/go-build"
GOENV="/home/oceanfish81/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/oceanfish81/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/oceanfish81/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/tools"
GCCGO="/usr/local/bin/llvm-goc"
AR="ar"
CC="/usr/bin/clang"
CXX="/usr/bin/clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build618217749=/tmp/go-build -gno-record-gcc-switches -funwind-tables"

That's under ubuntu 20 x86_64.

CC @bifurcation @suhasHere
BBC @thanm @cherrymui

Ivan

We need to verify interop with test vectors https://github.com/mlswg/mls-implementations

There are a bunch of panic statements scattered around right now. These should probably mostly be replaced with error return values, so that we only panic when it is really appropriate for the program to crash.

We should add test coverage measurement to CI, e.g., using Coveralls.

The current API is an uncomfortable mix of private (lower case) and public (upper case) APIs. "Uncomfortable" because, for example, there are public functions and struct fields that use private types.

We should follow some consistent theory about what is public and private. The two obvious alternatives:

1. Everything is public

• Pro: Allows for lower-level experimentation
• Con: Makes it harder for consumers to understand what APIs they should use

2. Only a "blessed" API is public; everything else is private

• Pro: Easier for consumers to use
• Con: Requires careful thought about what goes into the blessed API

Right now, our APIs use a mix of pointers and direct object references. We should be more uniform in our approach here. Suggested guidelines:

• For structs representing messages (e.g., Welcome, Commit), fields are by value, except when optional
• For other structs (e.g., State, HPKEPrivateKey), fields are as produced / passed to APIs, thus usually pointers
• Fields in messages are by value, except when optional
• Objects are passed to/from API methods by reference (as pointers)

• Remove blanking of nodes on Add (*)
• Change epoch numbers to uint64 (*)
• Add PSK inputs (*)
• Add key schedule exporter (*)
• Sign the updated direct path on Commit, using "parent hashes" and one signature per leaf (*)
• Use structured types for external senders (*)
• Redesign Welcome to include confirmation and use derived keys (*)
• Remove ignored proposals (*)
• Always include an Update with a Commit (*)
• Add per-message entropy to guard against nonce reuse (*)
• Use the same hash ratchet construct for both application and handshake keys (*)
• Add more ciphersuites
• Use HKDF to derive key pairs (*)
• Mandate expiration of ClientInitKeys (*)
• Add extensions to GroupContext and flesh out the extensibility story (*)
• Rename ClientInitKey to KeyPackage