substra / hlf-k8s Goto Github PK
View Code? Open in Web Editor NEWInitializes an Hyperledger Fabric network (orchestrator distributed mode)
Home Page: https://docs.substra.org
License: Apache License 2.0
Initializes an Hyperledger Fabric network (orchestrator distributed mode)
Home Page: https://docs.substra.org
License: Apache License 2.0
Hi!
I trying to start hlf on my kubernetes cluster on OpenStack with csi-sc-cinderplugin for PVC.
Some pods succesfully started (like couchdb, ca).
Other pods is stuck in ContainerCreating status. I've got typical errors from these pods like:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 48m default-scheduler Successfully assigned hlf-dev/hlf-hlf-k8s-toolbox-775bd96b6f-xph7m to k8s5
Warning FailedMount 48m (x2 over 48m) kubelet MountVolume.SetUp failed for volume "ord-tls-rootcert" : secret "ord-tls-rootcert" not found
Warning FailedMount 48m (x2 over 48m) kubelet MountVolume.SetUp failed for volume "id-key" : secret "hlf-msp-key-user" not found
Warning FailedMount 48m (x2 over 48m) kubelet MountVolume.SetUp failed for volume "cacert" : secret "hlf-cacert" not found
Warning FailedMount 48m (x2 over 48m) kubelet MountVolume.SetUp failed for volume "admin-key" : secret "hlf-msp-key-admin" not found
Warning FailedMount 48m (x2 over 48m) kubelet MountVolume.SetUp failed for volume "tls-client" : secret "hlf-tls-user" not found
Warning FailedMount 48m (x3 over 48m) kubelet MountVolume.SetUp failed for volume "tls-rootcert" : secret "hlf-cacert" not found
Warning FailedMount 48m (x2 over 48m) kubelet MountVolume.SetUp failed for volume "admin-cert" : secret "hlf-msp-cert-admin" not found
Warning FailedMount 42m (x10 over 48m) kubelet MountVolume.SetUp failed for volume "id-cert" : secret "hlf-msp-cert-user" not found
Warning FailedMount 38m (x8 over 48m) kubelet MountVolume.SetUp failed for volume "tls-clientrootcert" : secret "hlf-cacert" not found
Warning FailedMount 22m (x19 over 48m) kubelet MountVolume.SetUp failed for volume "tls" : secret "hlf-tls-admin" not found
Warning FailedMount 3m31s (x109 over 48m) kubelet (combined from similar events): Unable to attach or mount volumes: unmounted volumes=[id-key cacert tls-clientrootcert tls-rootcert tls-client id-cert ord-tls-rootcert admin-key tls admin-cert], unattached volumes=[id-key cacert tls-clientrootcert kube-api-access-x4kqv tls-rootcert tls-client id-cert ord-tls-rootcert admin-key fabric-config tls admin-cert]: timed out waiting for the condition
Please help me to debug & resolve.
Hello, if I use these helm charts to deploy my network, how can I extend my existed network? For example If I need to add a second channel (which will be joined by all the existing orgs) and deploy a new chaincode to this channel (or update the chaincode of an old channel). Can I do that somehow?
Finally, the gateway client functionality can also be deployed to k8s?
Hi there,
hlf-ca.orderer.host
variable seems unused am I right?
For stability purposes I'm trying to run the substra platform on microk8s instead
of minikube
(on ubuntu 18.04.5
). Using the configuration that was working with the latter:
docker 19.03.12
kubernetes 1.15.11
hlf-k8s 0.0.12
substra-backend 0.0.19
skaffold 1.8.0
dns
, ingress
, rbac
and helm 2.14.3
. I've added the bitnami
repository to helm, and for tiller to be successfuly setup on the cluster I've setup a service account
and a role binding
and applied it to the cluster using microk8s.kubectl apply -f my-setup.yaml
.skaffold run
from my hlf-k8s folder I end up with the following error:FATA[0001] failed to build: build failed: building [substrafoundation/hlf-k8s]: build artifact: denied: requested access to the resource is denied
I've tried to docker logout && docker login
, and also to do a helm repo update
with no success. And when I tried to install the chart manualy using microk8s.helm install --name orderer --namespace orderer substra/hlf-k8s --version 1.3.0 -f ./charts/hlf-k8s/values.yaml
I get the following error:
Error: failed to download "substra/hlf-k8s" (hint: running `helm repo update` may help)
Is it possible to add new org to an already deployed network?
Hello do we have an example for this?
Update [02-18-2022]
I've been experimenting with deploying multiple peers for an org in the same namespace. Using the existing configuration with external chaincode, im able to deploy a peer 2 to the same namespace and use the same chaincode pod that was deployed as part of the peer 1 release.
There's a modification that is required here. For chaincode operator, each release will deploy a chaincode deployment and operator if there's a chaincode object from values.yaml. The current implementation packages a new cc package(for each release) and this creation of package causes a potential conflict of the same intended chaincode to be used across the org.
I have a working example that will allow peer 2 to use the same chaincode package from a secret that was published from peer1's release.
I think this design is optimal as you can export the secret to import it across clusters if you have peers in other clusters to use.
Will add a PR when I get a chance but any feedback on this approach is appreciated OR if there's already an existing method from current implementation that I had missed would save me some effort.
I'm trying to run hlf-k8s using the latest line on the compatibility table. I'm running Ubuntu Server 18.05.4, with the following software versions:
1.9.2
19.03.13
1.15.11
0.0.13
0.0.11
1.8.0
3.4.1
1.18.0
I've tried using a local (by removing the chaincode[0].src
entry in the skaffold.yaml
file, and setting chaincodes[0].hostPath
to the absolute path to the chaincode repository) and a remote (by setting chaincode[0].src
to https://github.com/SubstraFoundation/substra-chaincode/archive/0.0.11.tar.gz
in the skaffold.yaml
file) version of the chaincode repository.
I've changed the hlf-k8s chart's requirements.yaml
file to replace https://kubernetes-charts.storage.googleapis.com
with https://charts.helm.sh/stable
, and then ran a
helm dependency update $absolute_path_to_chart
I then run the skaffold run
command and here's the error I get from the org-1/network-org-1-peer-1-hlf-peer
pod:
[endorser] callChaincode -> INFO 223 [yourchannel][4ba51601] Entry chaincode: name:"lscc"
[lscc] executeDeployOrUpgrade -> ERRO 224 cannot get package for chaincode (mycc:1.0)-err:open /var/hyperledger/production/chaincodes/mycc.1.0: no such file or directory
[endorser] callChaincode -> INFO 225 [yourchannel][4ba51601] Exit chaincode: name:"lscc" (1ms)
[endorser] ProcessProposal -> ERRO 226 [yourchannel][4ba51601] simulateProposal() resulted in chaincode name:"lscc" response status 500 for txid: 4ba516018a438e897a69671558f35ea65f6fa339ee943a7eee0ed930458325fe
[comm.grpc.server] 1 -> INFO 227 unary call completed grpc.service=protos.Endorser grpc.method=ProcessProposal grpc.peer_address=172.17.0.26:42370 grpc.peer_subject="CN=user,OU=peer,O=Hyperledger,ST=North Carolina,C=US" grpc.code=OK grpc.call_duration=1.501797ms
Hello there ๐
I tried today to make hlf-k8s works behind a proxy but it seems like I need to do a lot of changes in order to make it work.
First, configuring a forward proxy usually means setting a set of environment variables (HTTP_PROXY / NO_PROXY / HTTPS_PROXY etc.) but the chart doesn't allow to put extraEnv on the pods.
Then, some commands won't use the HTTP_PROXY configuration, like netcat (it doesn't use http protocol) so that means that this for example won't work: https://github.com/SubstraFoundation/hlf-k8s/blob/master/charts/hlf-k8s/templates/deployment-appchannel-operator.yaml#L52
Finally, after some digging it seems that the Hyperledger Fabric community discourage this setup and suggest using NAT / vpn instead:
I agree that it doesn't make sense for HLF to be behind a proxy (as introducing a proxy kind of break the trust in the network anyway), but, is it even doable ? What are your thoughts on that ?
Loading configuration
Loaded configuration: /substra/data/orgs/chu-nantes/configtx.yaml
[common.tools.configtxgen.encoder] NewConsortiumOrgGroup
WARN Default policy emission is deprecated, please include policy specifications for the orderer org group chu-nantes in configtx.yaml
/substra/conf/config/conf-chu-nantes.json
Will getChannelConfigBlockWithOrderer
got ChannelConfigBlockWithOrderer
signAndPushSystemUpdateProposal
Generating channel configuration transaction at /substra/data/channel/substrachannel.tx
....
channel creation: True
Wait For Peers to join channel
Join channel substrachannel with peers ['peer1-chu-nantes', 'peer2-chu-nantes'] ...
fail to get genesis block
Will retry to make peers join channel
fail to get genesis block
Will retry to make peers join channel
Peers ['peer1-chu-nantes', 'peer2-chu-nantes'] successfully joined channel substrachannel
Installing chaincode on ['peer1-chu-nantes', 'peer2-chu-nantes'] ...
policy: OR('chu-nantesMSP.member')
Instantiated chaincode with policy: {'identities': [{'role': {'name': 'member', 'mspId': 'chu-nantesMSP'}}], 'policy': {'1-of': [{'signed-by': 0}]}} and result: "timeout expired while starting chaincode substracc:1.0 for transaction"
Try to query chaincode from peer ['peer1-chu-nantes', 'peer2-chu-nantes'] on org chu-nantes
Traceback (most recent call last):
File "/scripts/run.py", line 135, in
add_org()
File "/scripts/run.py", line 116, in add_org
if client.queryChaincodeFromPeers() == '[]':
File "/scripts/utils/run_utils.py", line 283, in queryChaincodeFromPeers
cc_name=self.chaincode_name,
File "/usr/lib/python3.6/asyncio/base_events.py", line 484, in run_until_complete
return future.result()
File "/usr/local/lib/python3.6/dist-packages/hfc/fabric/client.py", line 1662, in chaincode_query
raise Exception(res)
Exception: [response {
status: 500
message: "make sure the chaincode substracc has been successfully instantiated and try again: chaincode substracc not found"
}
, response {
status: 500
message: "make sure the chaincode substracc has been successfully instantiated and try again: chaincode substracc not found"
}
]
There are currently no instructions on how to upgrade the chaincode version on a running installation.
With https://helm.sh/docs/topics/charts/#schema-files, we need to warn user if they deploy hlf-k8s without persistence over peer, orderer, ca, couchdb !
When owkin/charts#23 is merged we need to update the helm dep :)
I've tried changing the yaml configuration files to run the cluster with more then one peer. I've changed the skaffold.yaml
file as follows:
network-org-1-peer-1
:chaincodes[0].policy: OR("MyOrg1MSP.member"\,"MyOrg2MSP.member"\,"MyOrg3MSP.member")
network-org-1-peer-1
config:channels[0].extraOrgs[1].name: MyOrg3
channels[0].extraOrgs[1].secret: org-3-org-config-anchor
fetchSecrets[2].from: org-config-anchor
fetchSecrets[2].to: org-3-org-config-anchor
fetchSecrets[2].filename: configOrgWithAnchors.json
fetchSecrets[2].namespace: org-3
- name: network-org-3-peer-1
chartPath: charts/hlf-k8s
namespace: org-3
imageStrategy:
helm: {}
values:
image: substrafoundation/hlf-k8s
setValues:
nginx-ingress.enabled: true
nginx-ingress.controller.scope.enabled: true
ca.caName: rcaOrg3
peer.host: network-org-3-peer-1.org-3 # {name}.{namespace}
organization.id: MyOrg3MSP
organization.name: MyOrg3
peer.peer.mspID: MyOrg3MSP
chaincodes[0].name: mycc
chaincodes[0].version: "1.0"
# Note: Instead of an URL, you can use an absolute path, e.g. /home/johndoe/code/substra-chaincode
# This path folder must be accessible to kubernetes. See README for details.
chaincodes[0].src: https://github.com/SubstraFoundation/substra-chaincode/archive/master.tar.gz
channels[0].name: mychannel
channels[0].join: true
orderer.host: network-orderer.orderer
peer.peer.gossip.externalEndpoint: network-org-3-peer-1.org-3:7051 # {name}.{namespace}:{port}
fetchSecrets[0].from: ord-tls-rootcert
fetchSecrets[0].to: ord-tls-rootcert
fetchSecrets[0].filename: cacert.pem
fetchSecrets[0].namespace: orderer
What happens is that org-1 creates the channel, then the first peer to finish its setup between org-2 and org-3 manages to join it, but then the last peer can't because the handshake fails.
Here's the error I get from the network-org-1-peer-1-hlf-k8s-add-org-mychannel-myorg2
job:
Error: got unexpected status: BAD_REQUEST -- error applying config update to existing channel 'mychannel': error authorizing update: error validating DeltaSet: policy for [Group] /Channel/Application not satisfied: implicit policy ev aluation failed - 1 sub-policies were satisfied, but this policy requires 2 of the 'Admins' sub-policies to be satisfied
Testing the connection with the orderer:Testing the connection with the orderer:Testing the connection with the orderer:
And here's the error I get from the network-org-2-peer-1-hlf-k8s-channel-join-0
job:
Error: failed to create deliver client: orderer client failed to connect to network-orderer.orderer:7050: failed to create new connection: connection error: desc = "transport: authentication handshake failed: remote error: tls: bad certificate"
This condition returns true if the channel has joined a channel name which includes the target channel name.
For instance, if the peer has joined demochannel2
, then the condition will return true for demochannel
even though the peer hasn't joined demochannel
The 9443 port is used by Kuberenetes in docker for mac (precisely kube-compose-api-server)
So in a docker-compose setup, if kuberenetes is enabled, starting hlf-k8s will fail.
Is it possible to bind another port or to make prometheus optional by default ?
peer channel fetch config
writes its output inside / for some operator.
Hello there,
Currently you can make a peer join or configure only one channel with the chart. It would be nice, like it's done for the chaincodes, to be able to configure multiple.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.