square / subzero Goto Github PK

Block's Bitcoin Cold Storage solution.

License: Apache License 2.0

Go 0.18% Python 2.37% Shell 2.73% Java 35.32% CMake 2.17% Dockerfile 0.11% C 54.60% HTML 0.70% CSS 0.10% JavaScript 1.69% Kotlin 0.03%

bitcoin cold-storage hsm hardware-security-module btc crypto cryptocurrency

subzero's Introduction

HSM Cold Storage

For security, Square stores a reserve of Bitcoins in an offline setting. By having these funds offline, we reduce attack surface and risk of theft.

Square's solution is unique, specifically, we leverage FIPS certified Hardware Security Modules (HSMs) to protect the private key material. We decided to use such HSMs because we already own, operate, and trust these devices for other payment-related needs.

Funds can be sent from online systems to the cold storage at any time. Moving funds out of cold storage requires a multi-party signing ceremony. In addition, the offline HSMs are able to enforce business logic rules, for instance we only allow sending funds to Square-owned addresses. Such a scheme is usually called defense in depth or an onion model. We maintain the online/offline isolation by importing transaction metadata and exporting signatures using QR codes.

HSMs have the ability to share key material. This enables the ability to store our backups in encrypted form and restore a wallet at any location.

This repo contains our design documents as well as specific technical information. We are sharing our source code, with the caveat that the code is currently only useful if you have the exact same hardware setup. We are willing to make the code more modular over time, as long as the broader community shows interest to implement support for additional hardware vendors.

Documentation

https://subzero.readthedocs.io

License

Copyright 2018 Square, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

   http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

subzero's People

Contributors

Stargazers

Watchers

Forkers

lookfirst richardji7 d3vil7 coderfi frrp stjordanis hronopay jonnycrunch shaunstanislauslau mansaj devhttps drsm ilyeong-ai mikalv 2015091 kinggs tomzhang nikileshsa simonsarris dandycheung yzernik liutz vtproduction saruultugs davidgrinberg throughnothing absinthdelmar aaronkyle morristech alexintosh greg2git 8001800 mimoo duncwa heapxlabs flavioimbertdomingos monkey-mas ch1p999 qqonline incede seif-abaza kenyouuuu chao-huang jbelke andozw cnxtech mandlamag crypto501 jlopp lcheng62 mccrayinc levyforchh crazyk031 aussiehash leanderdulac ties dave7280 chhsiegel cambricorp test00dezwebsite huamichaelchen isabella232 jnaulty allen0cipher alkariuki jacobjohansen majacq snrt10081 ineba gubaojian puwaka 0xbool sts0mrg0 moza88 kaawatea nmahendru amankr1279 invis2912 hannibal0x charley-zhang ivmaykov prasanth08 aces-source classicvalues dcashman iq-scm ymuako skalarfeilds79 diversepwmeasurement iceblockware

subzero's Issues

init script assumes /hd/local does not exist

We should fail gracefully if /hd/local exists for any reason.

Fix readthedocs.io TOC rendering

Compare the left pane on from readthedocs.io (https://i.imgur.com/t4f97Z6.png) vs a local build (https://i.imgur.com/GH27swLr.png)

We should figure out why readthedocs is rendering the left pane incorrectly (ordering is incorrect and the chapters are flattened) or switch to a different way to build/distribute the docs.

For now, I recommend using mkdocs serve locally.

Remove k (R) from protobuf

We currently use RFC 6979, but we can make it a forcing factor by removing k (aka R) from the protobuf. This will make it harder to accidentally not implement RFC 6979.

Hook up linter/static analyzer/fuzzer

We should consider hooking up common tools, such as findbugs, etc.

Mitigate risks around smart card use

The C code (running in the HSM) could:

use the insertion counter to enforce one transaction signing operating per card insertion.
look into preventing key ticket re-use. We could perhaps send the card passwords to the C code instead of creating key tickets in the first place?

The risk we are mitigating is compromised Java code talking to the trusted C code.

Project Logo/icon?

Hey Folks,

This project is very impressive, and is likely the start of a great standard.

Have you considered putting a bit of effort into the branding side of things? A simple logo would be a great start.

Key tickets can be re-used

We should look into removing the need for key tickets or make them one-time use only.

Build Failure

CURRENCY=btc-testnet cmake ../

fails because subdirectory trezor-crypto is missing.

Document how to use nCipher in dev over SSH

HSM initialization page should explain how to:

create a wrapper script
set port forwarding
select nCipherKM.jar lib (has to be in the right directory) over the fake ncipher
set /opt/kmdata/local on the dev machine so that it's in sync with nCipher machine
set required config options in the YAML file

kleptograms / subliminal channels in ECDSA

The following two links discuss subliminal channels in ECDSA.

We should check if there's anything we can do in Subzero to defend against these.

https://media.ccc.de/v/35c3-9492-wallet_security and https://bitslog.com/2014/06/09/deterministic-signatures-subliminal-channels-and-hardware-wallets/

Proof wallet initialization happened correctly

The coordinator cannot tell if a wallet was initialized correctly or not. There are various edge cases (one site initialization multiple wallets, one person initializing the wallet on a non-HSM device, etc.)

One fix would be to use per-HSM RSA keys.

For now, we preferred to keep things simple and mitigate this risk by having out-of-band controls. We also perform a tiny transaction to ensure things were setup right.

Document when/how to use fake_ncipher module vs nCipherKM.jar

fake_ncipher module can make it hard to develop new features (since we only have symbols for the nCipher bits we currently use). We should document how to use nCipherKM.jar in dev and how to add missing symbols to fake_ncipher.

Hardcoded CodeSafe filename and version

bootstrap.sh includes a hardcoded version of CodeSafe Developer Kit ISO (CodeSafe-linux64-dev-12.50.2.iso). It should probably be listed in docs and/or parametrized.

Document compatible HSM hardware models

Hi Team

Would be great if information about the HSM spec that is more detailed than "Thales nCipher Solo XC low" is added to the documentation. Searching for "Thales nCipher Solo XC low" does not yield an exact match (there's "Thales nShield" but without knowing the specs it's hard to compare).

Does not have to be an exact model number, just which HSM devices this code is compatible with or if there is a way to test for compatibility without having to purchase the HSM would be great.

Thanks!

Document HSM keys/key material

We should document all the HSM keys/key material/passwords. Off the top of my head:

wallet ACS: needed to enroll new HSMs. Allows recovery of wallet OCS.
wallet OCS: allows decrypting wallet files to move funds.
signing ACS + OCS: needed at build time to sign the codesafe.
wallet files: contain the encrypted seed. The keys which encrypt wallet files are OCS protected and can only be used with signed codesafe.
softcard: used to protect the wallet's xpubs.
softcard password: isn't sensitive because the softcard protected key can only be used inside codesafe.

Document use of fake nCipher vs real nCipher jar

We don't explain when the fake nCipherKM.jar should be left in place and when the real one should be used. Might be worth explaining why we need the fake nCipher.

see-stdioesock-serve failing when running subzero

@alokmenghrajani any idea what would be causing this?

cc @lacombar

% /opt/nfast/bin/see-stdioesock-serv --machine subzero-unsigned.ar --userdata-raw userdata.cpio --trace
nC SEE bsdlib machine booting
AF_UNIX bound to hardserver emulation.
Hardserver IO thread started.
AF_UNIX using hardserver emulation for /opt/nfast/sockets/nserver
/proc mounted.
SEE job device IO thread started.
bsdsee nshield (12.63.0-63-43553d3)
ncseevfs_hostcall_open( "/dev/seejobs/6e432f48432f7374646f6520", ... )
setup provision /dev/seejobs/6e432f48432f7374646f6520 hostcall
:outstand_want = 4
:outstand_max = 8
allocate 100 spare tags
spare tags ready
provision /dev/seejobs/6e432f48432f7374646f6520 hostcall ready
/dev/seejobs/ job with unhandled prefix
 !!jobs=1 0ctrl [6e432f48432f7374646f6520]
 job starts [6e432f48432f737464696f65]...
*** abort() called

Fix --init-nvram in Java code for 2-of-N ACS quorums

--init-nvram works fine in dev (because I used 1-of-1 ACS quorum) but is a little buggy when the ACS quorum is more 2 or more.

We should fix the code + ensure we always dev/test with 2-of-N going forward.

Use a single proto file for C and Java

The contents of https://github.com/square/subzero/tree/master/core/proto/squareup/subzero and https://github.com/square/subzero/tree/master/java/proto/src/main/proto/squareup/subzero can probably be merged. Perhaps under common/proto/squareup/subzero?

Historically, we had these different directories because the code lived in different repos.

Disable unix users and make DVD a kiosk-like app

The DVD currently exposes the unix shell. We should make the app auto-launch and ensure you can't get a shell / can't login as any of the unix users.

Document supply chain

The docs don't explain what kinds of supply chain attacks we are worried about, what we are doing to mitigate them, what risks we need to assume, etc.

There's also many ways to approach this problem, and it might be worth clarifying our take / vision. For example, we believe that being able to reproduce builds is more useful than having a few people perform a clean build.

Running Electrum

cd electrum
cd contrib/build-osx

# only need to build once
./make_osx
cd ../../

# you can then edit the code in lib/ and see the changes
./electrum --testnet -v

The command to run electrum is

./electrum_run --testnet -v

NullpointerException

I'm trying to follow through with
https://subzero.readthedocs.io/en/master/running_without_hsm/

I managed to get to the point where i want to merge the signatures.
I have these values:

Initial sign transaction request: 
EAAqSQotCiC8fxRx/4QV6z/aar+xwiBzC76fynpmV+0DAtuZjqXbuRAAGJBOIgQQABgAEgsIqEYQAhoEEAAYABgAIgApAAAAAAAAAAA=

Initial finalize response 1: 
EnEKb3RwdWJEOGI1WXlROFZqQ3lUbmdIbzE5Y1pWQ0RrWTNiTVRxWmZQVWRneDJCaUREY3VkeXhmaE5pMTNIalhxbkw1TEVaWDVHSG10eUJ0MW9QbmdLWW5xQWJWWXY5WnRxRFROVlJjb1dlN0pYY1BXVw==

Initial finalize response 2: 
EnEKb3RwdWJEOHc0ZVRra25MYTJSVGJWbUc4U2I4UFdueW5LQm5vRWtRck1qSlg5bktEY2JlTUxMM0w4RHRvbnJyeTlydFlGQXlyZ29IdGZob3FDWUVrRkFOdFRiYlk5ZzdFVDdGQWRoZGhrVWc1dTVWdg==

Initial finalize response 3: 
EnEKb3RwdWJEOXRpcHN6b201cVh6NmloYkpmMXZzWFR1N3RMOVlHWDQ5ZU1RM1dnc044NnZ4RnRYTjJCTmNoZDdHYm1CYm9OSFFUSDNpYXBvdmZXRFdyemlLdHl4azhKQVJjY0FTempnMkd1Skp6aGhqRQ==

Initial finalize response 4: 
EnEKb3RwdWJEOVIyNml4Q2RrRXk0ZjVvVGVvbzlkOVBmenRtVXZGYnI1U0sxZ1ZTM3BOQndEMVZvdE00NEhHQjczMTE1WU55a0R4a2pONnF2RHNBYnVNYmhFa1BMVG9Qa3FnSEJmSFJuaGNRZGJxdGpjWA==

Gateway xpub: (from source):
tpubDA3zCqbkh8BS1h3VGmB6VU1Cj638VJufBcDWSeZVw3aEE9rzvmrNoKyGFqDwr8d9rhf4sh

Sign transaction response 1: 
Gm0KawpHMEUCIQDmTIgUC+MCGKSr8Fz8/CtpU3hNGIt6ldZbB+OgsmPJ0wIgbWfPwkPKUyYgp6TS9ZMJZ9DhiIvZnmlPpz1XNG6OtA4SIBh0uyQREYaNMD3GzcCLI30XNHRvZ4vJD8cUExNtO9wd

Sign transaction response 2: 
Gm0KawpHMEUCIQCk/lWb2mq5iy5FNpk+sJfQ4vWFv/yuMz7ysrBvYmUfOwIgG+mglKdgQG+7naY9HT9m9eSoRaGisf8guvGj/GXRPg8SIBh0uyQREYaNMD3GzcCLI30XNHRvZ4vJD8cUExNtO9wd

When i click on "show final transaction", i get a NullPointer:


127.0.0.1 - - [05/Nov/2019:13:09:19 +0000] "GET /show-final-transaction?signTxRequest=EAAqSQotCiC8fxRx%2F4QV6z%2Faar%2BxwiBzC76fynpmV%2B0DAtuZjqXbuRAAGJBOIgQQABgAEgsIqEYQAhoEEAAYABgAIgApAAAAAAAAAAA%3D&finalizeResponses=EnEKb3RwdWJEOGI1WXlROFZqQ3lUbmdIbzE5Y1pWQ0RrWTNiTVRxWmZQVWRneDJCaUREY3VkeXhmaE5pMTNIalhxbkw1TEVaWDVHSG10eUJ0MW9QbmdLWW5xQWJWWXY5WnRxRFROVlJjb1dlN0pYY1BXVw%3D%3D&finalizeResponses=EnEKb3RwdWJEOHc0ZVRra25MYTJSVGJWbUc4U2I4UFdueW5LQm5vRWtRck1qSlg5bktEY2JlTUxMM0w4RHRvbnJyeTlydFlGQXlyZ29IdGZob3FDWUVrRkFOdFRiYlk5ZzdFVDdGQWRoZGhrVWc1dTVWdg%3D%3D&finalizeResponses=EnEKb3RwdWJEOXRpcHN6b201cVh6NmloYkpmMXZzWFR1N3RMOVlHWDQ5ZU1RM1dnc044NnZ4RnRYTjJCTmNoZDdHYm1CYm9OSFFUSDNpYXBvdmZXRFdyemlLdHl4azhKQVJjY0FTempnMkd1Skp6aGhqRQ%3D%3D&finalizeResponses=EnEKb3RwdWJEOVIyNml4Q2RrRXk0ZjVvVGVvbzlkOVBmenRtVXZGYnI1U0sxZ1ZTM3BOQndEMVZvdE00NEhHQjczMTE1WU55a0R4a2pONnF2RHNBYnVNYmhFa1BMVG9Qa3FnSEJmSFJuaGNRZGJxdGpjWA%3D%3D&gateway=tpubDA3zCqbkh8BS1h3VGmB6VU1Cj638VJufBcDWSeZVw3aEE9rzvmrNoKyGFqDwr8d9rhf4sh&signTxResponses=Gm0KawpHMEUCIQDmTIgUC%2BMCGKSr8Fz8%2FCtpU3hNGIt6ldZbB%2BOgsmPJ0wIgbWfPwkPKUyYgp6TS9ZMJZ9DhiIvZnmlPpz1XNG6OtA4SIBh0uyQREYaNMD3GzcCLI30XNHRvZ4vJD8cUExNtO9wd&signTxResponses=Gm0KawpHMEUCIQCk%2FlWb2mq5iy5FNpk%2BsJfQ4vWFv%2FyuMz7ysrBvYmUfOwIgG%2BmglKdgQG%2B7naY9HT9m9eSoRaGisf8guvGj%2FGXRPg8SIBh0uyQREYaNMD3GzcCLI30XNHRvZ4vJD8cUExNtO9wd HTTP/1.1" 500 110 "http://localhost:8080/assets/index.html" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" 29
ERROR [2019-11-05 13:09:20,365] io.dropwizard.jersey.errors.LoggingExceptionMapper: Error handling a request: 511de5ed00cb2830
! java.lang.NullPointerException: null
! at org.bitcoinj.crypto.DeterministicKey.deserialize(DeterministicKey.java:547)
! at org.bitcoinj.crypto.DeterministicKey.deserializeB58(DeterministicKey.java:530)
! at org.bitcoinj.crypto.DeterministicKey.deserializeB58(DeterministicKey.java:521)
! at com.squareup.subzero.shared.ColdWallet.lambda$new$0(ColdWallet.java:53)
! at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
! at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1654)
! at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
! at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
! at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
! at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
! at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
! at com.squareup.subzero.shared.ColdWallet.<init>(ColdWallet.java:54)
! at com.squareup.subzero.server.resources.ShowFinalTransactionResource.request(ShowFinalTransactionResource.java:50)
! at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
! at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
! at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
! at java.base/java.lang.reflect.Method.invoke(Method.java:567)
! at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
! at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
! at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
! at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
! at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
! at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
! at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
! at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
! at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
! at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
! at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
! at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
! at io.dropwizard.jetty.NonblockingServletHolder.handle(NonblockingServletHolder.java:49)
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
! at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:35)
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
! at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:45)
! at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:39)
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
! at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
! at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
! at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
! at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
! at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
! at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
! at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:239)
! at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52)
! at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:724)
! at io.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:67)
! at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56)
! at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:169)
! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
! at org.eclipse.jetty.server.Server.handle(Server.java:531)
! at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
! at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
! at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
! at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
! at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
! at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
! at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
! at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
! at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
! at java.base/java.lang.Thread.run(Thread.java:835)

Any idea what might be wrong here? How can i track that down more efficiently?

Automatically update docs

Figure out how to configure readthedocs.io or travis build-github pages integration, or whatever.

Right now, the docs are on https://square.github.io/subzero/ but aren't automagically updated.

Implement better health checks in the GUI

When the GUI starts up, we should perform some basic checks and report a nice error message (or warning):

is there a wallet directory?
can we talk to the HSM?
can we talk to the CodeSafe? Is the CodeSafe's self check successful?
do we have the right number of keys?
do the keys + nvram ACLs look right?
etc.

validateCommandRequest should check QR code size

https://github.com/square/subzero/blob/master/java/shared/src/main/java/com/squareup/subzero/shared/SubzeroUtils.java#L295 does not check if the resulting request (and corresponding response) will fit in a QR code.

We have constants (https://github.com/square/subzero/blob/master/java/shared/src/main/java/com/squareup/subzero/shared/Constants.java#L30) we can use (or just calculate the request/response sizes).

Not able to build core

Hello I am getting error below on the make step trying to build the core component. Any ideas on what the issue could be?

C02WW21FHV2Q:build a530616101$ make
[  1%] Running C++ protocol buffer compiler using nanopb plugin on /Users/a530616101/Documents/c/subzero/core/proto/squareup/subzero/common.proto
[  2%] Running C++ protocol buffer compiler using nanopb plugin on /Users/a530616101/Documents/c/subzero/core/proto/squareup/subzero/internal.proto
Traceback (most recent call last):
  File "/Users/a530616101/Documents/c/subzero/core/build/nanopb/generator/nanopb_generator.py", line 1861, in <module>
    main_plugin()
  File "/Users/a530616101/Documents/c/subzero/core/build/nanopb/generator/nanopb_generator.py", line 1846, in main_plugin
    results = process_file(filename, fdesc, options, other_files)
  File "/Users/a530616101/Documents/c/subzero/core/build/nanopb/generator/nanopb_generator.py", line 1727, in process_file
    headerdata = ''.join(f.generate_header(includes, headerbasename, options))
  File "/Users/a530616101/Documents/c/subzero/core/build/nanopb/generator/nanopb_generator.py", line 1404, in generate_header
    yield msg.fields_declaration(self.dependencies) + '\n'
  File "/Users/a530616101/Documents/c/subzero/core/build/nanopb/generator/nanopb_generator.py", line 1028, in fields_declaration
    defval = self.default_value(dependencies)
  File "/Users/a530616101/Documents/c/subzero/core/build/nanopb/generator/nanopb_generator.py", line 1127, in default_value
    desc = google.protobuf.descriptor.MakeDescriptor(optional_only)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/google/protobuf/descriptor.py", line 1015, in MakeDescriptor
    _message.default_pool.Add(file_descriptor_proto)
TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "b3f05eb848476f7dca0c05a3400c4658.proto":
  InternalCommandRequest.FinalizeWalletRequest.encrypted_master_seed: ".EncryptedMasterSeed" is not defined.
  InternalCommandRequest.FinalizeWalletRequest.encrypted_pub_keys: ".EncryptedPubKey" is not defined.
  InternalCommandRequest.SignTxRequest.encrypted_master_seed: ".EncryptedMasterSeed" is not defined.
  InternalCommandRequest.SignTxRequest.encrypted_pub_keys: ".EncryptedPubKey" is not defined.
  InternalCommandRequest.SignTxRequest.inputs: ".TxInput" is not defined.
  InternalCommandRequest.SignTxRequest.outputs: ".TxOutput" is not defined.
  InternalCommandRequest.command: Oneof must have at least one field.

Simplify segwit code in java/

The segwit code in java/ predates Segwit support in BitcoinJ. We can probably clean some things up.

Improve launch script

The bash script which launches the codesafe module and the login process which blocks on it is unnecessary. We should leverage services instead. CentOS supports defining dependencies between services. We could also make the GUI smarter and have it show a status (e.g. hardware detected, waiting for module to load).

Make base64 decoding new line agnostic

Our base64 decoding is strict and fails if a string ends (or contains) a newline. Accidental newlines are common (mostly trailing new lines when using the debug server and copy-pasting data around). There's no harm in being new line agnostic.

What is recommended hardware for development/connecting to HSM

If I may ask, for development purposes, are you connecting the HSM via an external PCIe card enclosure or you actually have a laptop/server with enough space to fit an HSM?

Any model/make recommendations you can offer, without compromising your own security, would be much appreciated. Plenty of options to buy but would rather go with what is known to work.

BIP 69

We should document that we don't do anything special related to BIP 69. Subzero keeps inputs/outputs in their original order.

It's the responsibility of the online system to handle UTXO management and also follow recommendations such as BIP 69. The online system should also implement best practices, such as not re-using addresses, handling dust, etc.

Update HSM instructions to use migrate-world instead of mk-reprogram

nCipher's v12.60.7 and newer no longer supports mk-reprogram. We need to use migrate-world instead. We should update our instructions accordingly and/or revisit the dual-security world setup.

Document running signed vs unsigned code

We should document how to run unsigned code (set UNSIGNED=yes when running cmake). We should also document the other parameters (I don't think we explain anywhere how TARGET=nCipher works).

Adding business rules

In the release blog post it was mentioned that the choice for the Thales HSM was in part due to the ability to enforce business rules with in the HSM custom code. I've been looking for said code (mainly here), but I'm unable to pinpoint the checking of the business rules. Where would I find this?

An unrelated question; how easy would it be to add other currencies?

Figure out how to compile HSM code on a modern compiler

The compiler we currently (are forced) to use doesn't have some modern protections (such as stack/heap protections).

We should investigate using a different compiler. Perhaps even re-writing some pieces in a safer language such as Rust.

Improve UI/layout code

The graphical UI code was an afterthought (the initial design was a command line tool). As a result, the code doesn't use the right design patterns (e.g. there's no MVC). We also place text at fixed offset which only works for a single screen resolution (and assumes the system font is a specific height).

We should consider either:

improving the layout code to be screen resolution / font agnostic.
rendering AWT to a frame buffer.
running X/Wayland and have the Java code be a normal applet.
some other alternative.

A better UI code would make adding new screens easier (such as backup/export wallet, audit ACLs, etc.)

A related area for improvement is error handling. E.g. making sure we give a clear message if the C code isn't running or if it failed its self checks.

issues trying to build/boot dvd

@alokmenghrajani unclear what the centos root username and password are, is it just root with one of the passwords in https://github.com/square/subzero/blob/master/live-usb-creator/rhel7-livemedia.ks?

Also noticed the top of that file says "DEVEL" - outside of changing username/password, anything you'd recommend to be changed for production?

Originally posted by @bosswissam in #14 (comment)

Single ISO for dev/staging/prod

Having to build different ISOs for different envs is a pain. We should have a single ISO which works everywhere.

build errors at core/

I followed the steps below based on https://subzero.readthedocs.io/en/master/running_without_hsm/:

The final "make" command failed with some directory missing errors.

build the C code, using testnet transactions.

cd ../core
mkdir build
cd build
TARGET=dev CURRENCY=btc-testnet cmake ../
make

This-MacBook-Pro:build vagrant$ make
[ 1%] Generating ../nanopb/generator/proto/plugin_pb2.py
/Users/vagrant/work/test2/subzero/core/build/nanopb/generator/proto: warning: directory does not exist.
Could not make proto path relative: /Users/vagrant/work/test2/subzero/core/build/nanopb/generator/proto/plugin.proto: No such file or directory
make[2]: *** [nanopb/generator/proto/plugin_pb2.py] Error 1
make[1]: *** [proto/CMakeFiles/PlutusProtos.dir/all] Error 2
make: *** [all] Error 2

I also tried the solution of #35,

git fetch origin pull/37/head:pull_37
git checkout pull_37
git submodule update --init --recursive

However, it doesn't help.

I'm not using docker build. It is just a vagrant mac build.

Hope someone can help.

Potential post bitcoinj 0.15 cleanup

#94 bumps the version of bitcoinj to 0.15.2.

It might now be possible to delete createWrappedSegwitMultisigTransaction. This was needed as a workaround due to the lack of Segwit support in bitcoinj -- which was added in 0.15.

note: we might still need bits and pieces of createWrappedSegwitMultisigTransaction if Bitcoinj doesn't support segwit + multisig, but we might be able to at least simplify things a little.

Shade java dependencies

Subzero's java dependencies can be hard to upgrade in the event they conflict with other systems.

The interface between Subzero and other systems is mostly just SubzeroUtils.java.

Shading the java dependencies might make upgrades easier.

Provide a way to audit the state of things

It might be useful to have a way to audit the state of keys + an indication if anything isn't as expected.

show Codesafe module is signed. List ACLs on the signing key.
show nvram ACLs.
show ACLs/algorithms for various keys.

dev-without-hsm = no HSM, btc-testnet
dev-with-hsm = HSM, unsigned, btc-testnet
staging = HSM, signed, btc-testnet
prod = HSM, signed, btc-mainnet

See https://github.com/square/subzero/pull/78/files#r265746401 for context.