etsangsplk / blockchain-provisioning Goto Github PK

This project forked from circlesubi/blockchain-provisioning

Scripts & documentation used to provision a private PoA Ethereum blockchain

HCL 81.78% Shell 6.06% Dockerfile 0.70% Python 11.46%

blockchain-provisioning's Introduction

DEPRECATED

The circles pilot will be run on Kovan instead of a private testnet. This repo exists only for reference / archival purposes and does not describe a live system anywhere.

Blockchain Provisioning

This repository contains terraform scripts defining an AWS environment containing a private Ethereum blockchain

Using the Cluster

Monitor the current status of the cluster with our ethstats instance at: stats.circles-chain.com
Connect metamask to the RPC node at: http://35.158.153.37:8545

Running a Local Node

Run one of these from the project root. NODE_NAME is the name that will be shown at stats.circles-chain.com

Full node: ./nodes/full <NODE_NAME>
Light client: ./nodes/light <NODE_NAME>

Bringing Up the AWS environment

Install terraform
Run aws configure --profile circles-blockchain-provisioning
Enter the access-key and secret, and set the default region as eu-central-1. The output format can be None.
terraform init
terraform apply

Operating the Cluster

aside from a few exceptions (see below) this git repository should fully and completly describe the cluster and all associated AWS infrastructure
updates to the cluster should only be applied by changing the source code and running terraform apply
instances are immutable (never modified after they are deployed)
if an instance needs to be changed then it will be destroyed and an updated replacement will be built from source and deployed

Manually Provisioned Resources

Terraform State and Locking Table

state is persisted to an encrypted & versioned S3 bucket
in order to reduce the risk that a bad commit could destroy the state the required resources (S3 bucket + dynamoDB table) are managed outside of Terraform
specified in providers.tf
docs: state | backends

Secrets

due to the risk of leaking state, all sensistive information should be managed outside of Terraform
Stored in Secrets Manager
Pulled onto servers with a python script using IAM roles defined in iam.tf

EFS Folder Structure

Directories in EFS can only be created by mounting an instance and running mkdir.
Nodes share the same EFS volume but only mount subdirectories. Subdirectories cannot be mounted until they actually exist.
EFS volume needs /sealer and /rpc directories (should be empty)

Environment / Topology

Data Persistance

defined in storage.tf
all state is persisted in a single EFS filesystem volume

Consensus Parameters

defined in resources/genesis.json
proof of Authority chain using geth with Clique.
5s block times

Network

defined in network.tf
1 private & 1 public subnet in eu-central-1

Logging

cloud-init logs for each instance written to cloudwatch logs

Services

Each service runs on a single burstable t2.micro instance (defined in services/base.tf).

sealer (x2)

produces blocks
holds private keys
running geth
private subnet

rpc

rpc ports opened to public internet
relays blocks to sealer
allows interaction with metamask
running geth
public subnet

bootnode

service discovery
requires open udp ports to the network
running geth
public subnet

ethstats

monitoring dashboard for the cluster
running eth-netstats
public subnet

TODO

Big

Block explorer -> blocked by blockscout/blockscout#196
Define staging environment & deployment pipeline
Bootnode should be stateful
Authentication for RPC endpoints
Define (and ideally automate) secret rotation procedures
Backup chain state (ideally some versioned / snapshotting system). Test restoration from these backups.