I am trying to setup all-in-one management hub in a ubuntu 18.04 machine when I got this issue. I tried stopping and restarting using the flags -S and -s, and also I tried stopping and purging using -SP flags then setting it up again, but unfortunately without success. The logs do show an issue of not able to initialize the agbot:

2021-10-07T15:51:22.705263071Z I1007 15:51:22.705147 10 main.go:65] Using config: Edge: {ServiceStorage , APIListen 127.0.0.1:8510, DBPath , DockerEndpoint , DockerCredFilePath , DefaultCPUSet , DefaultServiceRegistrationRAM: 0, StaticWebContent: , PublicKeyPath: , TrustSystemCACerts: false, CACertsPath: , ExchangeURL: , DefaultHTTPClientTimeoutS: 30, PolicyPath: , ExchangeHeartbeat: 0, AgreementTimeoutS: 0, DVPrefix: , RegistrationDelayS: 0, ExchangeMessageTTL: 0, ExchangeMessageDynamicPoll: true, ExchangeMessagePollInterval: 20, ExchangeMessagePollMaxInterval: 120, ExchangeMessagePollIncrement: 20, UserPublicKeyPath: , ReportDeviceStatus: false, TrustCertUpdatesFromOrg: false, TrustDockerAuthFromOrg: false, ServiceUpgradeCheckIntervalS: 300, MultipleAnaxInstances: false, DefaultServiceRetryCount: 2, DefaultServiceRetryDuration: 600, NodeCheckIntervalS: 15, FileSyncService: {APIListen: , APIPort: 0, APIProtocol: , PersistencePath: , AuthenticationPath: , CSSURL: , CSSSSLCert: , PollingRate: 0, ObjectQueueBufferSize: 0}, InitialPollingBuffer: {120}, BlockchainAccountId: , BlockchainDirectoryAddress }, AgreementBot: {TxLostDelayTolerationSeconds: 120, AgreementWorkers: 5, DBPath: , Postgresql: {Host: postgres, Port: 5432, User: admin, Password: ******, DBName: exchange, SSLMode: disable MaxOpenConnections: 20}, PartitionStale: 0, ProtocolTimeoutS: 120, AgreementTimeoutS: 360, NoDataIntervalS: 300, ActiveAgreementsURL: , ActiveAgreementsUser: , ActiveAgreementsPW: ******, PolicyPath: /home/agbotuser/policy.d/, NewContractIntervalS: 5, ProcessGovernanceIntervalS: 5, IgnoreContractWithAttribs: ethereum_account, ExchangeURL: http://exchange-api:8080/v1/, ExchangeHeartbeat: 5, ExchangeId: IBM/agbot, ExchangeToken: ******, DVPrefix: , ActiveDeviceTimeoutS: 180, ExchangeMessageTTL: 1800, MessageKeyPath: msgKey, DefaultWorkloadPW: ******, APIListen: 0.0.0.0:8080, SecureAPIListenHost: 0.0.0.0, SecureAPIListenPort: 8083, SecureAPIServerCert: , SecureAPIServerkey: , PurgeArchivedAgreementHours: 1, CheckUpdatedPolicyS: 7, CSSURL: http://css-api:8080, CSSSSLCert: , AgreementBatchSize: 300, AgreementQueueSize: 300, MessageQueueScale: 33, QueueHistorySize: 30, FullRescanS: 600, MaxExchangeChanges: 1000, RetryLookBackWindow: 3600, PolicySearchOrder: true, Vault: {{http://vault:8200 }}}, Collaborators: {HTTPClientFactory: &{0x7556a0 0 10}, KeyFileNamesFetcher: &{0x755e90}}, ArchSynonyms: {map[aarch64:arm64 armhf:arm x86_64:amd64]}
2021-10-07T15:51:22.705524354Z I1007 15:51:22.705481 10 main.go:66] GOMAXPROCS: 1
2021-10-07T15:51:22.705587108Z I1007 15:51:22.705563 10 init.go:22] Connecting to Postgresql database: host=postgres port=5432 user=admin dbname=exchange sslmode=disable
2021-10-07T15:51:22.708502419Z I1007 15:51:22.708469 10 init.go:40] Agreementbot 7fd4aea5-6a4d-4924-a59f-33e11d3b5fd5 initializing partitions
2021-10-07T15:51:22.708554453Z I1007 15:51:22.708535 10 init.go:43] Postgresql database tables initializing.
2021-10-07T15:51:22.736060867Z panic: Unable to initialize Agreement Bot: unable to claim a partition, error: unable to claim an unowned partition, error: unable to claim stale, error: pq: query is not a SELECT
2021-10-07T15:51:22.736113111Z
2021-10-07T15:51:22.736330618Z goroutine 1 [running]:
2021-10-07T15:51:22.736503923Z main.main()
2021-10-07T15:51:22.736722773Z /tmp/anax-gopath/src/github.com/open-horizon/anax/main.go:92 +0x210b

If this is the issue, what might be the problem? Your help is highly appreciated!

Just now deploy-mgmt-hub.sh script supports amd64 platform only. Is it possible to add support for Power (ppc64le) platform? Maybe without SDO stuff that requires amd64 only. Also RHEL/CentOS support is desired because both often are used on this platform.

I installed open horizon, it was working fine but I forgot to save HZN_ORG_ID and HZN_EXCHANGE_USER_AUTH. Then I restarted my system, after restart I was unable to make connection with horizon api (getting error). I tried reinstalling it but I was getting same error in re-installation.

Error: Can't connect to the Horizon REST API to run GET http://localhost:8510/node. Run 'systemctl status horizon' to check if the Horizon agent is running. Or set HORIZON_URL to connect to another local port that is connected to a remote Horizon agent via a ssh tunnel. Specific error is: Get "http://localhost:8510/node": dial tcp 127.0.0.1:8510: connect: connection refused

Output of command "systemctl status horizon":

horizon.service - Service for Horizon control system
Loaded: loaded (/lib/systemd/system/horizon.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2021-01-15 00:45:54 IST; 15min ago
Process: 23471 ExecStart=/usr/horizon/bin/anax -v $ANAX_LOG_LEVEL -logtostderr -config /etc/horizon/anax.json (code=exited, status=98)
Main PID: 23471 (code=exited, status=98)

Jan 15 00:45:54 user-Vostro-3546 systemd[1]: horizon.service: Service hold-off time over, scheduling restart.
Jan 15 00:45:54 user-Vostro-3546 systemd[1]: horizon.service: Scheduled restart job, restart counter is at 6.
Jan 15 00:45:54 user-Vostro-3546 systemd[1]: Stopped Service for Horizon control system.
Jan 15 00:45:54 user-Vostro-3546 systemd[1]: horizon.service: Start request repeated too quickly.
Jan 15 00:45:54 user-Vostro-3546 systemd[1]: horizon.service: Failed with result 'exit-code'.
Jan 15 00:45:54 user-Vostro-3546 systemd[1]: Failed to start Service for Horizon control system.

• Have the script automatically add a hub admin using a curl command similar to:

curl -sS -w %{http_code} -u "root/root:${EXCHANGE_ROOT_PW:?}" -X POST -H Content-Type:application/json -d '{"hubAdmin":true,"admin":false,"password":"abc","email":""}' $HZN_EXCHANGE_URL/orgs/root/users/hubadmin | jq

• Manually test that hubadmin can create an org using hzn exchange org create ...
• Add example of adding an org in the "What's Next" section of documentation

Hi

Hit into following error when executed either ./deploy-mgmt-hub.sh or curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash
`root@hub:/home/oliver# ./deploy-mgmt-hub.sh
Management hub services will listen on http://127.0.0.1
Updating apt-get package index...
Installing prerequisites, this could take a minute...
----------- Downloading template files...
Substituting environment variables into template files...
----------- Downloading/starting Horizon management hub services...
Downloading management hub docker images...
Pulling openhorizon/amd64_agbot:latest ...
Pulling openhorizon/amd64_exchange-api:latest ...
Pulling openhorizon/amd64_cloud-sync-service:latest ...
Pulling postgres:latest ...
Pulling mongo:latest ...
Pulling openhorizon/sdo-owner-services:latest ...
Starting management hub containers...
Traceback (most recent call last):
File "urllib3/connectionpool.py", line 677, in urlopen
File "urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1277, in request
File "http/client.py", line 1323, in _send_request
File "http/client.py", line 1272, in endheaders
File "http/client.py", line 1032, in _send_output
File "http/client.py", line 972, in send
File "docker/transport/unixconn.py", line 43, in connect
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "requests/adapters.py", line 449, in send
File "urllib3/connectionpool.py", line 727, in urlopen
File "urllib3/util/retry.py", line 410, in increment
File "urllib3/packages/six.py", line 734, in reraise
File "urllib3/connectionpool.py", line 677, in urlopen
File "urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1277, in request
File "http/client.py", line 1323, in _send_request
File "http/client.py", line 1272, in endheaders
File "http/client.py", line 1032, in _send_output
File "http/client.py", line 972, in send
File "docker/transport/unixconn.py", line 43, in connect
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "docker/api/client.py", line 214, in _retrieve_server_version
File "docker/api/daemon.py", line 181, in version
File "docker/utils/decorators.py", line 46, in inner
File "docker/api/client.py", line 237, in _get
File "requests/sessions.py", line 543, in get
File "requests/sessions.py", line 530, in request
File "requests/sessions.py", line 643, in send
File "requests/adapters.py", line 498, in send
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "docker-compose", line 3, in
File "compose/cli/main.py", line 81, in main
File "compose/cli/main.py", line 200, in perform_command
File "compose/cli/command.py", line 70, in project_from_options
File "compose/cli/command.py", line 153, in get_project
File "compose/cli/docker_client.py", line 43, in get_client
File "compose/cli/docker_client.py", line 170, in docker_client
File "docker/api/client.py", line 197, in init
File "docker/api/client.py", line 222, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
[6747] Failed to execute script docker-compose
Error: exit code 255 from: starting docker-compose services
root@hub:/home/oliver#`

The image remove command that PR 42 added was too broad:

runCmdQuietly docker rmi $(docker images openhorizon/* -q)

Because the developer could have his/her own openhorizon images he/she doesn't want deleted. Also, it was resulting in a bunch of errors like this on my mac:

Error response from daemon: conflict: unable to delete 855debc7d8e0 (must be forced) - image is referenced in multiple

Also, it wasn't documented in the usage -P would also remove images.

In the All-in-1 readme, the instructions mention that this should be able to run in Mac. However, after running sudo -i and then curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash I get this result.

----------- Verifying input and the host OS...
Management hub services will listen on 127.0.0.1
Installing prerequisites using brew, this could take a minute...
Error running brew install jq gettext: Error: Running Homebrew as root is extremely dangerous and no longer supported.
As Homebrew does not drop privileges on installation you would be giving all
build scripts full access to your system.

A workaround is to brew install all the necessary packages before running sudo -i. I exited sudo -i, ran brew install jq gettext, entered sudo again, and the install went further before I encountered this message.

Configuring the Horizon agent and CLI...
Starting the Horizon agent container...
You must have socat installed to run this command.
Install socat using homebrew: http://macappstore.org/socat/, or using MacPorts: https://www.macports.org/ then 'sudo port install socat'
Error: exit code 2 from: starting agent

After exiting from sudo -i and running brew install socat, I was able to run sudo -i once more and then was able to successfully install the All-in-1.

Ideally, I think the script should be refactored to somehow get around that limitation. A shorter term solution is to update the instructions to pre-install the packages I listed above. I understand that this might be low priority since the Mac instructions are considered experimental, but I'm willing to try taking this on to ensure others have a smoother experience getting set up with Open Horizon 😃

If it helps, I'm running macOS Catalina Version 10.15.5.

Mini-design:

• TLS only supported on linux (not macos)
• Change http: to ${HZN_TRANSPORT}:
• if HZN_TRANSPORT==https create key/cert in /etc/horizon/keys/horizonMgmtHub.{crt,key} (if doesn't exist yet):

      openssl req -newkey rsa:4096 -nodes -sha256 -x509 -keyout $CERT_DIR/$CERT_BASE_NAME.key -days 365 -out $CERT_DIR/$CERT_BASE_NAME.crt -subj "/C=US/ST=NY/L=New York/[email protected]/CN=$(hostname)" -extensions san -config <(echo '[req]'; echo 'distinguished_name=req'; echo '[san]'; echo "subjectAltName=IP:$allIPs")
  

  • ln -sf horizonMgmtHub.crt sdoapi.crt && ln -sf horizonMgmtHub.key sdoapi.key #todo: add to sdo env vars for the key paths
  • ln -s /etc/horizon/keys/horizonMgmtHub.crt /etc/ssl/certs ??
• if HZN_TRANSPORT==http remove the keys/certs (so mounts in docker-compose.yml will mount empty dir)
• Have purge delete the key/cert
• Set HZN_MGMT_HUB_CERT_PATH=$CERT_DIR/$CERT_BASE_NAME.crt in /etc/default/horizon and agent-install.cfg
• Add cert as agent-install.crt to CSS
• At end of script, output export HZN_MGMT_HUB_CERT_PATH=$CERT_DIR/$CERT_BASE_NAME.crt
• Agbot:
  • Set in agbot-tmpl.json:

      "SecureAPIServerCert": "/home/agbotuser/keys/horizonMgmtHub.crt",
      "SecureAPIServerKey": "/home/agbotuser/keys/horizonMgmtHub.key",
  

  • Mount /etc/horizon/keys to /home/agbotuser/keys
• SDO:
  • Mount /etc/horizon/keys to /home/sdouser/ocs-api-dir/keys
• CSS:
  • Set in css-tmpl.conf:

      ListeningType secure
      SecureListeningPort 8080
      ServerCertificate /home/cssuser/keys/horizonMgmtHub.crt
      ServerKey /home/cssuser/keys/horizonMgmtHub.key
  

  • Mount /etc/horizon/keys to /home/cssuser/keys
• (wait until it supports TLS) Exchange:
  • change other container's references to exchange
• Postgres & Mongo:
  • Neither of these expose any ports on the host, so don't need to TLS them at this point

Original request from @JoergWende :

Hi,

I've tried to set https as the communication protocol but the CSS container does not start with:
Invalid communication protocol, please choose either HTTP or MQTT or WIoTP

Before digging any deeper - is there a standard way to run a secure infrastructure ?

Cheers
J.

Provide a switch/env-var that causes the manage hub script to skip publishing the example services from the OH examples repo. This will reduce the amount of time it takes for the hub to be started.

Hi. I am trying to setup all-in-one management hub in a Ubuntu 18.04.4 and 20.04.3 LTS (Virtual box). My issues with command for run deploy-mgmt-hub.sh, it doesn't working correctly, how I see docker containers was created and started, but edge node wasn't created.

The logs Ubuntu 20.04.3 LTS:

root@ubuntuoh:~ curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash
------- Checking input and initializing...
Management hub services will listen on http://127.0.0.1
Updating apt-get package index...
Installing prerequisites, this could take a minute...
Docker is required, installing it...
OK
Hit:1 http://ua.archive.ubuntu.com/ubuntu focal InRelease
Hit:2 http://ua.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:3 http://ua.archive.ubuntu.com/ubuntu focal-backports InRelease
Hit:4 http://ua.archive.ubuntu.com/ubuntu focal-security InRelease
Get:5 https://download.docker.com/linux/ubuntu focal InRelease [57.7 kB]
Get:6 https://download.docker.com/linux/ubuntu focal/stable amd64 Packages [11.6 kB]
Fetched 69.3 kB in 1s (109 kB/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
docker-ce-rootless-extras docker-scan-plugin pigz slirp4netns
Suggested packages:
aufs-tools cgroupfs-mount | cgroup-lite
The following NEW packages will be installed:
containerd.io docker-ce docker-ce-cli docker-ce-rootless-extras docker-scan-plugin pigz slirp4netns
0 upgraded, 7 newly installed, 0 to remove and 28 not upgraded.
Need to get 95.6 MB of archives.
After this operation, 403 MB of additional disk space will be used.
Get:1 http://ua.archive.ubuntu.com/ubuntu focal/universe amd64 pigz amd64 2.4-1 [57.4 kB]
Get:2 http://ua.archive.ubuntu.com/ubuntu focal/universe amd64 slirp4netns amd64 0.4.3-1 [74.3 kB]
Get:3 https://download.docker.com/linux/ubuntu focal/stable amd64 containerd.io amd64 1.4.11-1 [23.7 MB]
Get:4 https://download.docker.com/linux/ubuntu focal/stable amd64 docker-ce-cli amd64 5:20.10.93-0ubuntu-focal [38.8 MB]
Get:5 https://download.docker.com/linux/ubuntu focal/stable amd64 docker-ce amd64 5:20.10.93-0ubuntu-focal [21.2 MB]
Get:6 https://download.docker.com/linux/ubuntu focal/stable amd64 docker-ce-rootless-extras amd64 5:20.10.93-0ubuntu-focal [7,914 kB]
Get:7 https://download.docker.com/linux/ubuntu focal/stable amd64 docker-scan-plugin amd64 0.8.0ubuntu-focal [3,889 kB]Fetched 95.6 MB in 5s (21.1 MB/s)
Selecting previously unselected package pigz.
(Reading database ... 71545 files and directories currently installed.)
Preparing to unpack .../0-pigz_2.4-1_amd64.deb ...
Unpacking pigz (2.4-1) ...
Selecting previously unselected package containerd.io.
Preparing to unpack .../1-containerd.io_1.4.11-1_amd64.deb ...
Unpacking containerd.io (1.4.11-1) ...
Selecting previously unselected package docker-ce-cli.
Preparing to unpack .../2-docker-ce-cli_5%3a20.10.93-0ubuntu-focal_amd64.deb ...
Unpacking docker-ce-cli (5:20.10.93-0ubuntu-focal) ...
Selecting previously unselected package docker-ce.
Preparing to unpack .../3-docker-ce_5%3a20.10.93-0ubuntu-focal_amd64.deb ...
Unpacking docker-ce (5:20.10.93-0ubuntu-focal) ...
Selecting previously unselected package docker-ce-rootless-extras.
Preparing to unpack .../4-docker-ce-rootless-extras_5%3a20.10.93-0ubuntu-focal_amd64.deb ...
Unpacking docker-ce-rootless-extras (5:20.10.93-0ubuntu-focal) ...
Selecting previously unselected package docker-scan-plugin.
Preparing to unpack .../5-docker-scan-plugin_0.8.0ubuntu-focal_amd64.deb ...
Unpacking docker-scan-plugin (0.8.0ubuntu-focal) ...
Selecting previously unselected package slirp4netns.
Preparing to unpack .../6-slirp4netns_0.4.3-1_amd64.deb ...
Unpacking slirp4netns (0.4.3-1) ...
Setting up slirp4netns (0.4.3-1) ...
Setting up docker-scan-plugin (0.8.0ubuntu-focal) ...
Setting up containerd.io (1.4.11-1) ...
Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service.Setting up docker-ce-cli (5:20.10.93-0ubuntu-focal) ...
Setting up pigz (2.4-1) ...
Setting up docker-ce-rootless-extras (5:20.10.93-0ubuntu-focal) ...
Setting up docker-ce (5:20.10.93-0ubuntu-focal) ...
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /lib/systemd/system/docker.socket.
Processing triggers for man-db (2.9.1-1) ...
Processing triggers for systemd (245.4-4ubuntu3.11) ...
docker-compose is not installed or not at least version 1.21.0, installing/upgrading it...
######################################################################## 100.0%##O=# # ######################################################################## 100.0%#-#O=# # ######################################################################## 100.0%
------- Downloading template files...
Substituting environment variables into template files...
------- Downloading/starting Horizon management hub services...
Downloading management hub docker images...
Pulling openhorizon/amd64_agbot:latest ...
Pulling openhorizon/amd64_exchange-api:latest ...
Pulling openhorizon/amd64_cloud-sync-service:latest ...
Pulling postgres:latest ...
Pulling mongo:latest ...
Pulling openhorizon/sdo-owner-services:latest ...
Pulling openhorizon/amd64_vault:latest ...
Starting management hub containers...
[+] Running 13/13
⠿ Network hzn_horizonnet Created 0.3s
⠿ Volume "hzn_agbotmsgkeyvol" Created 0.0s
⠿ Volume "hzn_postgresvol" Created 0.0s
⠿ Volume "hzn_mongovol" Created 0.0s
⠿ Volume "hzn_ocsdb" Created 0.0s
⠿ Volume "hzn_vaultvol" Created 0.0s
⠿ Container mongo Started 6.0s
⠿ Container postgres Started 6.1s
⠿ Container exchange-api Started 4.5s
⠿ Container css-api Started 9.2s
⠿ Container sdo-owner-services Started 8.3s
⠿ Container vault Started 9.1s
⠿ Container agbot Started 11.9s
Waiting for the exchange.............
------- Creating the user org, and the admin user in both orgs...
Creating exchange hub admin user, and the admin user and agbot in the system org...
------- Creating a Vault instance and preforming all setup and configuration operations ...
Checking Vault service status, initialization, and seal...
A Vault instance has not been initialized. Initializing...
Vault instance is sealed. Unsealing...
Creating KV ver.2 secrets engine openhorizon...
Registering auth plugin openhorizon-exchange to Vault instance...
Generating SHA256 hash of openhorizon-exchange plugin...
Enabling auth method openhorizon-exchange for secrets engine openhorizon...
Configuring auth method openhorizon-exchange for use with the Exchange...
Success! Data written to: auth/openhorizon/config
root@ubuntuoh:# hzn node list
hzn: command not found
root@ubuntuoh:# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a745bfc14e96 openhorizon/amd64_agbot:latest "/bin/sh -c /usr/hor…" 18 minutes ago Restarting (2) 47 seconds ago agbot
0da9eb60d590 openhorizon/amd64_cloud-sync-service:latest "/usr/edge-sync-serv…" 18 minutes ago Up 6 minutes (unhealthy) 127.0.0.1:9443->8080/tcp css-api
f3777e743bbf openhorizon/sdo-owner-services:latest "/bin/sh -c $WORKDIR…" 18 minutes ago Up 18 minutes (healthy) 127.0.0.1:8040->8040/tcp, 127.0.0.1:8042->8042/tcp, 127.0.0.1:9008->9008/tcp sdo-owner-services
6d38e46f0e99 openhorizon/amd64_vault:latest "entrypoint.sh server" 18 minutes ago Up 18 minutes (healthy) 127.0.0.1:8200->8200/tcp vault
3b8b4ef7533d openhorizon/amd64_exchange-api:latest "/bin/sh -c '/usr/bi…" 18 minutes ago Up 18 minutes (healthy) 8083/tcp, 127.0.0.1:3090->8080/tcp exchange-api
37a5230b6522 mongo:latest "docker-entrypoint.s…" 18 minutes ago Restarting (132) 52 seconds ago mongo
838a0541cf5a postgres:latest "docker-entrypoint.s…" 18 minutes ago Up 18 minutes (healthy) 5432/tcp postgres
root@ubuntuoh:~

As a part of Issue #43 create a Proof of Concept implementation for two of the options
* Implement compilation of Anax code (Go dependencies)
* Ensure that we have an environment that can handle the e2edev test code (ability to run Docker containers, etc). Note that the PoC does not necessarily need to run the e2edev tests but at least needs to be able to support it.
* Should take approximately 2 weeks each for PoC implementation

In the following section: https://github.com/open-horizon/devops/blob/master/mgmt-hub/README.md#-pausing-the-services you show how to stop the services. The instructions assume that the person has a local copy of the script. However, the one-liner that we used to install and run the services did not save a local copy. You may fix that by adding the following to the instructions:

Save a local executable copy of the script by running:

curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh > deploy-mgmt-hub.sh
chmod a+x deploy-mgmt-hub.sh

In the all-in-1 environment, a few utilities would help to automate the development process of building the modified source code of 1 of the components (agbot, agent, css, exchange, sdo) and restarting the service with it.

• HZN_AGBOT_URL in /etc/default/horizon (completed several weeks ago in a separate issue)

The related Anax issue is open-horizon/anax#2465

When I am try to install the all-in-one env in my Mac, I met this error :

sh-3.2# curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash
----------- Verifying input and the host OS...
Management hub services will listen on 127.0.0.1
----------- Downloading template files...
Substituting environment variables into template files...
----------- Downloading/starting Horizon management hub services...
Downloading management hub docker images...
Pulling openhorizon/amd64_agbot:latest...
Pulling openhorizon/amd64_exchange-api:latest...
Pulling openhorizon/amd64_cloud-sync-service:latest...
Pulling postgres:latest...
Pulling mongo:latest...
Pulling openhorizon/sdo-owner-services:latest...
Starting management hub containers...
mongo is up-to-date
postgres is up-to-date
Recreating exchange-api ... done
Recreating sdo-owner-services ... done
Recreating css-api            ... done
Recreating agbot              ... done
Waiting for the exchange
----------- Creating the user org, the admin user in both orgs, and an agbot in the exchange...
Creating exchange admin user and agbot in the system org...
Creating exchange user org and admin user...
----------- Downloading/installing Horizon agent and CLI...
Downloading the Horizon agent and CLI packages...
Installing the Horizon CLI package...
installer: Package name is horizon-cli-2.27.0-173
installer: Upgrading at base path /
installer: The upgrade was successful.
Configuring the Horizon agent and CLI...
Restarting the Horizon agent container...
Stopping/removing the horizon container...
horizon1
horizon1
testing: Pulling from openhorizon/amd64_anax
Digest: sha256:fd84578b134ff65b58288514056492254acff716842a48bf7edef658421863b0
Status: Image is up to date for openhorizon/amd64_anax:testing
docker.io/openhorizon/amd64_anax:testing
Starting the Horizon agent container openhorizon/amd64_anax:testing...
c7ca6dc47b9c713e730dce57d20d6a7ed4bbb7b68c605212b9efdde674229a52
Horizon agent updated/restarted successfully.
----------- Creating developer key pair, and installing Horizon example services, policies, and patterns...
Using examples repo branch v2.27 derived from the hzn version
Cloning https://github.com/open-horizon/examples.git to /tmp/open-horizon/examples ...
Error running git clone https://github.com/open-horizon/examples.git /tmp/open-horizon/examples: Cloning into '/tmp/open-horizon/examples'...
error: RPC failed; curl 18 transfer closed with outstanding read data remaining
fatal: the remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
Error: exit code 128 from: publishing examples

For agbot, we need to set ANAX_LOG_LEVEL env var for the agbot container.

Setting the environment variable AGBOT_ID to something other than agbot does not work. The agbot id in the exchange is always 'agbot'

These lines need to use the variable $AGBOT_ID instead of hardcoded agbot

A long standing issue in OpenShift causes a variety of issues, generally resulting in the non-root user being unable to read or write any files at all.

The relevant YAML files can be found here: https://github.com/illeatmyhat/devops/tree/k8s/mgmt-hub/k8s

This manifests in the management hub services as
agbot:

Error: container create failed: time="2021-07-02T09:37:28-05:00" level=error msg="container_linux.go:366: starting container process caused: chdir to cwd (\"/home/agbotuser\") set in config.json failed: permission denied"

css:

/usr/edge-sync-service/bin/css_start.sh: line 3: /etc/edge-sync-service/sync.conf: Permission denied
/usr/edge-sync-service/bin/css_start.sh: line 4: /home/cssuser/cloud-sync-service: Permission denied

exchange:

/bin/sh: /etc/horizon/exchange/config.json: Read-only file system

sdo:

Error: container create failed: time="2021-07-02T09:51:15-05:00" level=error msg="container_linux.go:366: starting container process caused: chdir to cwd (\"/home/sdouser\") set in config.json failed: permission denied"

The databases, mongo and postgres, have dedicated OpenShift images which already work.
Particularly, they seem to take the solution of changing the ownership and permissions of relevant directories: https://github.com/sclorg/postgresql-container/blob/generated/13/root/usr/libexec/fix-permissions

I was following the "Deploy All-in-1 Horizon Management Hub, Agent, and CLI" steps in the devops readme for Ubuntu 18.04.
I ran into an issue during the "Downloading template files" step:

Error: http code 000 from: downloading OH_DEVOPS_REPO/mgmt-hub/vault-tmpl.json, stderr: curl: (6) Could not resolve host: OH_DEVOPS_REPO

Looks like this could be avoided by bash-style interpreting the OH_DEVOPS_REPO variable during the template file download. I would be happy to make this change.

Based on chosen CI build option to replace Travis-CI, create a build pipeline that can compile Anax and run the e2edev tests
Should integrate with GitHub Pull Requests. So anytime a PR is created for Anax, it should trigger this build pipeline and serve as a gate to a PR merge. Once this is working in your own fork, you will need the help of an administrator to change the Anax GitHub repo settings.

Zhe Yan reported on the LF Edge open-horizon-mgmt-hub channel (on matrix chat svr) that on his Ubuntu 18.04.3 LTS on x86 machine, docker-compose is panicking:

~# curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash
Management hub services will listen on http://127.0.0.1
Updating apt-get package index...
Installing prerequisites, this could take a minute...
----------- Downloading template files...
Substituting environment variables into template files...
----------- Downloading/starting Horizon management hub services...
Downloading management hub docker images...
Pulling openhorizon/amd64_agbot:latest ...
Pulling openhorizon/amd64_exchange-api:latest ...
Pulling openhorizon/amd64_cloud-sync-service:latest ...
Pulling postgres:latest ...
Pulling mongo:latest ...
Pulling openhorizon/sdo-owner-services:latest ...
Starting management hub containers...
Traceback (most recent call last):
File "urllib3/connectionpool.py", line 677, in urlopen
File "urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1277, in request
File "http/client.py", line 1323, in _send_request
File "http/client.py", line 1272, in endheaders
...

The python version of docker-compose is only supposed to be used on ppc64le, but in some corner cases the DOCKER_COMPOSE_CMD variable gets set incorrectly.

When a PR is created, a GitHub Action runs the e2edev test against the new code. Once this is successful, the code should be merged in and a new GitHub Action should be kicked off that builds Anax and creates the Docker images. These images should be uploaded into Dockerhub.

Output of this Issue should be to create the second GH Action that builds and uploads the docker images. You can use your own fork for development and upload into your own Dockerhub private repo for testing. Once the code is finished and working, it should be changed to use the OpenHorizon org's Dockerhub repo.

Investigate and research open source CI technologies to migrate the Open-Horizon projects to from using Travis-CI.

Possible Options

1. LF-Edge Jenkins & Jenkins Job Builder Plugin
2. GitHub Actions
3. CircleCI - Research to see if there is a free option for Open Source projects

Output:

• Fork and create a branch in the open-horizon/devops repo
• Create a Pros/Cons for each one in MarkDown format in a new folder called documentation
  • will need to create criteria list to judge the tools on

the install script uses cat to create the default config files:

mkdir -p /etc/horizon   # putting the config files here because they are mounted long-term into the containers
cat $TMP_DIR/exchange-tmpl.json | envsubst > /etc/horizon/exchange.json
cat $TMP_DIR/agbot-tmpl.json | envsubst > /etc/horizon/agbot.json
cat $TMP_DIR/css-tmpl.conf | envsubst > /etc/horizon/css.conf

which means that the users umask will determine the permissions.

in my case it meant that they were not world readable, causing the exchange-api and agbot to fail.

I would suggest adding:

chmod a+r /etc/horizon/*

after those lines

Need to improve Horizon packages installation logics in RHEL on ppc64le platform avoiding packages deletion. Such deletion potentially could affect already running hzn services and/or Docker containers that have Horizon packages dependency. Also need to restrict support of RHEL to 8.x versions only as a mainstream in Open Horizon now.

I ran the deploy_mgmt_hub.sh script for the first time on a MacBook Pro with a recent OSX. Near the end of the install process, the script checks to see if socat is installed, and attempts to install it if it is not present. Since the install script is running as root using sudo, this can be problemmatic with some package managers.

In this particular case, it appears to be trying to use MacPorts to install socat. This fails on my MacBook because MacPorts is not installed. It recommends sudo port install socat which fails because "port" is not on my machine.

I do have homebrew installed, so if I drop out of sudo back to a user account and run brew install socat and then sudo -i to become root again, and then run ./deploy_mgmt_hub.sh again, it works.

I'm not sure if the solution here is to check for socat at the beginning of the script and tell OSX users to install socat manually if it does not exist, or if there is a better solution. I just thought you'd want to know that it does not currently work for users in my situation on fresh OSX installs.

Currently as is described in the README.md: the script deploy-mgmt-hub.sh is only supported for Ubuntu 18.x and macOS.
It would be nice to extend the support to CentOS so that I can deploy the all-in-1 Horizon Management Hub, Agent and CLI on my Intel NUC6CAYB running CentOS Stream 8.

I love the all-in-one script, but I want to take it further!

I've already made a cut-down version of the script (deploy-node.sh) to just install an agent and at some point in the future will want a deploy-cluster.sh too.

Plus flags to not actually run anything on the node.

Would this group be interested in such scripts, and would it be preferable to have them in a single script (a generic deploy.sh with lots of flags) or multiple?

cheers,
Andreas

After the installation, the node's configstate is configured:

hzn node list
{
  "id": "node1",
  "organization": "myorg",
  "pattern": "",
  "name": "node1",
  "nodeType": "device",
  "token_last_valid_time": "2020-11-30 16:25:02 +0800 CST",
  "token_valid": true,
  "ha": false,
  "configstate": {
    "state": "configured",
    "last_update_time": "2020-11-30 16:25:02 +0800 CST"
  },
  "configuration": {
    "exchange_api": "http://host.docker.internal:3090/v1/",
    "exchange_version": "2.56.0",
    "required_minimum_exchange_version": "2.44.0",
    "preferred_exchange_version": "2.44.0",
    "mms_api": "http://host.docker.internal:9443",
    "architecture": "amd64",
    "horizon_version": "2.27.0-173"
  }
}

However, I have stopped and started it, the state is unconfigured:

huleis-mbp:mgmt-hub root# ./deploy-mgmt-hub.sh -S
Unregistering this node, cancelling all agreements, stopping all workloads, and restarting Horizon...
Waiting for Horizon node unregister to complete: No Timeout specified ...
Waiting for Horizon node unregister to complete: No Timeout specified ...
Waiting for agent service to restart and checking the node configuration state...
Horizon node unregistered. You may now run 'hzn register ...' again, if desired.
Stopping the Horizon agent...
Unregistering the node, then stopping/removing the horizon container (this may take a minute)...
horizon1
horizon1
horizon1_var
horizon1_etc
Killing socat PIDs: 36507...
Stopping Horizon management hub services...
Stopping agbot              ... done
Stopping css-api            ... done
Stopping sdo-owner-services ... done
Stopping exchange-api       ... done
Stopping postgres           ... done
Stopping mongo              ... done
Removing agbot              ... done
Removing css-api            ... done
Removing sdo-owner-services ... done
Removing exchange-api       ... done
Removing postgres           ... done
Removing mongo              ... done
Removing network hzn_horizonnet
huleis-mbp:mgmt-hub root# ./deploy-mgmt-hub.sh -s
Starting management hub containers...
Pulling openhorizon/amd64_agbot:latest...
Pulling openhorizon/amd64_exchange-api:latest...
Pulling openhorizon/amd64_cloud-sync-service:latest...
Pulling postgres:latest...
Pulling mongo:latest...
Pulling openhorizon/sdo-owner-services:latest...
Creating network "hzn_horizonnet" with driver "bridge"
Creating postgres ... done
Creating mongo    ... done
Creating exchange-api ... done
Creating sdo-owner-services ... done
Creating css-api            ... done
Creating agbot              ... done
Starting the Horizon agent...
Starting socat to listen on port 2375 and forward it to the docker API socket...
testing: Pulling from openhorizon/amd64_anax
Digest: sha256:fd84578b134ff65b58288514056492254acff716842a48bf7edef658421863b0
Status: Image is up to date for openhorizon/amd64_anax:testing
docker.io/openhorizon/amd64_anax:testing
Starting the Horizon agent container openhorizon/amd64_anax:testing...
43be3b110277b3616e2f4eb2d92eca7156afabfe08186064b4f21503dc7236fc
Horizon agent started successfully. Now use 'hzn node list', 'hzn register ...', and 'hzn agreement list'
huleis-mbp:mgmt-hub root# hzn node list
{
  "id": "node1",
  "organization": null,
  "pattern": null,
  "name": null,
  "nodeType": null,
  "token_last_valid_time": "",
  "token_valid": null,
  "ha": null,
  "configstate": {
    "state": "unconfigured",
    "last_update_time": ""
  },
  "configuration": {
    "exchange_api": "http://host.docker.internal:3090/v1/",
    "exchange_version": "2.56.0",
    "required_minimum_exchange_version": "2.44.0",
    "preferred_exchange_version": "2.44.0",
    "mms_api": "http://host.docker.internal:9443",
    "architecture": "amd64",
    "horizon_version": "2.27.0-173"
  }
}

I am confuse to this, Could I know if it is OK. or Are there some missing in my side ? Thanks

Unregistration problem scenario:

deploy-mgmt-hub.sh   # edge node service agreement formed
deploy-mgmt-hub.sh   # during edge node unregistration it displays the error below
...
The node was not successfully unregistered, please use 'hzn unregister -D' to ensure the node is completely reset. Specific anax API error is: Can't connect to the Horizon REST API to run DELETE http://localhost:8510/node?block=true. Run 'systemctl status horizon' to check if the Horizon agent is running. Or run 'curl http://localhost:8081/status' to check the Horizon agent status. Or set HORIZON_URL to connect to another local port that is connected to a remote Horizon agent via a ssh tunnel. Specific error is: Delete "http://localhost:8510/node?block=true": EOF
...

Registration problem scenario:

deploy-mgmt-hub.sh   # edge node service agreement formed
hzn unregister -f   # to avoid the error above
deploy-mgmt-hub.sh   # no error msg, but no edge node service agreement formed

Notes added by @bmpotter :

• The root of the problem is the error that the agent encounters on macOS: Error starting containers: API error (500): error while creating mount source path '/private/var/tmp/horizon/horizon1/ess-auth/90e3023e2a14b2d6b27254b6331eb33cd890a4076b9c74d2fa4a9914a1d9b669': mkdir /private/var/tmp/horizon/horizon1/ess-auth/90e3023e2a14b2d6b27254b6331eb33cd890a4076b9c74d2fa4a9914a1d9b669: operation not permitted
• Even though it has been hard for the open-horizon dev team to recreate this problem, similar problems have been encountered by other users too (see anax issue 2017 ), so we need to get to the bottom of this.

Original description by @janvda:

When running the script deploy-mgmt-hub.sh on my macbook (macOS Big Sur) it seems to be properly installing the management hub but it doesn't succeed in starting the IBM/ibm.helloworld service.

FYI I have not used this version of the script but my version which contains a simple fix for issue #28

Here below the full log (I have masked the password).

Jans-MBP:open-horizon root# export HZN_LISTEN_IP=192.168.1.5
Jans-MBP:open-horizon root# ./deploy-mgmt-hub.sh
----------- Verifying input and the host OS...
Management hub services will listen on 192.168.1.5
----------- Downloading template files...
Substituting environment variables into template files...
----------- Downloading/starting Horizon management hub services...
Downloading management hub docker images...
Pulling openhorizon/amd64_agbot:latest...
Pulling openhorizon/amd64_exchange-api:latest...
Pulling openhorizon/amd64_cloud-sync-service:latest...
Pulling postgres:latest...
Pulling mongo:latest...
Pulling openhorizon/sdo-owner-services:latest...
Starting management hub containers...
Creating network "hzn_horizonnet" with driver "bridge"
Creating mongo    ... done
Creating postgres ... done
Creating exchange-api ... done
Creating css-api            ... done
Creating sdo-owner-services ... done
Creating agbot              ... done
Waiting for the exchange...
----------- Creating the user org, the admin user in both orgs, and an agbot in the exchange...
Creating exchange admin user and agbot in the system org...
Creating exchange user org and admin user...
----------- Downloading/installing Horizon agent and CLI...
Downloading the Horizon agent and CLI packages...
Installing the Horizon CLI package...
installer: Package name is horizon-cli-2.27.0-173
installer: Upgrading at base path /
installer: The upgrade was successful.
Configuring the Horizon agent and CLI...
Starting the Horizon agent container...
Starting socat to listen on port 2375 and forward it to the docker API socket...
testing: Pulling from openhorizon/amd64_anax
Digest: sha256:fd84578b134ff65b58288514056492254acff716842a48bf7edef658421863b0
Status: Image is up to date for openhorizon/amd64_anax:testing
docker.io/openhorizon/amd64_anax:testing
Starting the Horizon agent container openhorizon/amd64_anax:testing...
2301651cbf2c401f29be531af34e737af010614a33889a0547005125b410bd9e
Horizon agent started successfully. Now use 'hzn node list', 'hzn register ...', and 'hzn agreement list'
----------- Creating developer key pair, and installing Horizon example services, policies, and patterns...
Using examples repo branch v2.27 derived from the hzn version
Cloning https://github.com/open-horizon/examples.git to /tmp/open-horizon/examples ...
Switching to branch v2.27 ...
Branch 'v2.27' set up to track remote branch 'v2.27' from 'origin'.
Publishing services and patterns of edge/services/cpu_percent to IBM org...
Publishing services and patterns of edge/services/gps to IBM org...
Publishing services and patterns of edge/services/helloworld to IBM org...
Publishing deployment policy of edge/services/helloworld to myorg org...
Publishing services and patterns of edge/services/helloMMS to IBM org...
Publishing services and patterns of edge/services/hello-operator to IBM org...
Publishing deployment policy of edge/services/hello-operator to myorg org...
Publishing services and patterns of edge/evtstreams/cpu2evtstreams to IBM org...
Publishing deployment policy of edge/evtstreams/cpu2evtstreams to myorg org...
Successfully published all examples to the exchange. Removing /tmp/open-horizon/examples directory.
----------- Creating and registering the edge node with policy to run the helloworld Horizon example...
Waiting for the agent to be ready
Horizon Exchange base URL: http://192.168.1.5:3090/v1
Updating node token...
Will proceeed with the given node policy.
Updating the node policy...
Initializing the Horizon node with node type 'device'...
Note: no input file was specified. This is only valid if none of the services need variables set.
However, if there is 'userInput' specified in the node already in the Exchange, the userInput will be used.
Changing Horizon state to configured to register this node with Horizon...
Horizon node is registered. Workload services should begin executing shortly.
Waiting for up to 100 seconds for following services to start:
        IBM/ibm.helloworld
Status of the services you are watching:
        Waiting         IBM/ibm.helloworld
Status of the services you are watching:
        Waiting         IBM/ibm.helloworld
Status of the services you are watching:
        Waiting         IBM/ibm.helloworld
Status of the services you are watching:
        Waiting         IBM/ibm.helloworld
Status of the services you are watching:
        Waiting         IBM/ibm.helloworld
Status of the services you are watching:
        Waiting         IBM/ibm.helloworld
Timeout waiting for some services to successfully start. Analyzing possible reasons for the timeout...
The following services might need more time to start executing, continuing analysis:
        IBM/ibm.helloworld

Currently, there are 1 active agreements on this node. Use `hzn agreement list' to see the agreements that have been formed so far.

Analysis complete.

----------- Summary of what was done:
  1. Started Horizon management hub services: agbot, exchange, postgres DB, CSS, mongo DB
  2. Created exchange resources: system org (IBM) admin user, user org (myorg) and admin user, and agbot
     - Exchange root user generated password: 4Y....w
     - System org admin user generated password: i2f....M
     - Agbot generated token: sCl....iXi
     - User org admin user generated password: Ck9....aPY
     - Node generated token: nCZ.....umH
     Important: save these generated passwords/tokens in a safe place. You will not be able to query them from Horizon.
  3. Installed the Horizon agent and CLI (hzn)
  4. Created a Horizon developer key pair
  5. Installed the Horizon examples
  6. Created and registered an edge node to run the helloworld example edge service

For what to do next, see: https://github.com/open-horizon/devops/blob/master/mgmt-hub/README.md#all-in-1-what-next
Before running the commands in the What To Do Next section, copy/paste/run these commands in your terminal:
 export HZN_ORG_ID=myorg
 export HZN_EXCHANGE_USER_AUTH=admin:Ck9...LPY

I have managed to install Open Horizon on Ubuntu 20.04.2 with a hacked version of deploy-mgmt-hub.sh. Should I send in a PR for an updated version that runs on Ubuntu 20.x?

At time of writing, on all Open Horizon documentation I can find, only Ubuntu 18.04 and macOS are supported, but I believe we can use Ubuntu 20 without any issues (based on the reasoning that OH is very containerised). Also, I am willing to be a test subject for Ubuntu 20 compatibility... 🧑‍✈️ 🚀

I deployed an all-in-one, but no agreements ever formed, even though node registration succeeded. Looking at the agbot logs I saw lots of credential errors.

Looking inside the container, we discovered that the /etc/horizon/anax.json file had somehow gotten the wrong token! Editing this file to contain the correct token, and then stopping and starting the container fixed the problem. Agreements with registered nodes now happen as expected.

I noticed that this bind happens for the agbot container:

"Binds": [
                "/etc/horizon/agbot.json:/etc/horizon/anax.json.tmpl:ro"
...

Notice the .tmpl suffix. This file contains the correct token and is identical to the bound host file (as expected since it it s readonly bind). So I assume that the AgBot is supposed to copy this bound file into anax.json (without the .tmpl suffix) but for some reason it didn't just copy it; it edited the token, which broke the agbot.

User got the following error when running deploy-mgmt-hub.sh

----------- Downloading template files...
/tmp/deploy-mgmt-hub.sh: line 404: /tmp/horizon/curlExchangeErrors: Not a directory
Error: http code 200 from: /tmp/horizon/curlExchangeErrors
All-in-one management hub deployment: Failed deploy.
Makefile:438: recipe for target 'run-mgmthub' failed
make: *** [run-mgmthub] Error 1

This is because that there is a file called /tmp/horizon exists. deploy-mgmt-hub.sh command should tolerate it and delete it. Another suggestion is that it uses a non-general name like /tmp/horizon-mgmt-hub instead of /tmp/horizon to reduce the chances of conflicts.

This will make it easier to add additional edge nodes to the all-in-1 mgmt hub by using agent-install.sh -i anax: -k css:

Anax issue open-horizon/anax#2448 adds that new agent-install.sh flag combination.

Add an option for deploy-mgmt-hub.sh script so that it will install everything except the edge node. This option will be used in e2edev testing.

See open-horizon/anax#2404 and make essentially the same changes to the all in one management hub script, with the following clarifications:

1. for now, only start the vault when the HZN_VAULT environment variable is set to something/anything. In the future we will make starting the vault the default behavior, but not while the feature is being built.
2. the config file for the vault in the all in one hub should reside in the devops repo

Downloading the Horizon agent and CLI packages...
Error: http code 404 from: downloading https://github.com/open-horizon/anax/releases/latest/download/ubuntu.bionic.amd64.assets.tar.gz, stdout: Not Found

I used the readme:
https://github.com/open-horizon/devops/blob/master/mgmt-hub/README.md

It would be good to have the exchange-api starting timeout bigger that 20 seconds (at least 30 sec, maybe up to 1 minute), as it could be the problem for some users. For my case - I was able to fit into that timeout after 3 times of trying and each time I had
Error: can not reach the exchange at http://127.0.0.1:3090/v1 (tried for 20 seconds): curl: (52) Empty reply from server

There are a few improvements needed for all-in-1 to be used by e2edev:

• Image pull: Always pull when the image tag is latest or testing. For other tags, try to pull, but do not report error if the image exists locally, but does not exist in the remote repo. This way e2edev can use the locally built image.
• Add a second agbot when env variable START_SECOND_AGBOT is set to true.
• Add -c <config-file> to be able to pass in a set env vars
• Add HZN_AGBOT_URL and HZN_SDO_SVC_URL to /etc/default/horizon and agent-install.cfg
• Test/fix listening on 0.0.0.0 with external edge node
• Change Adding More Edge Nodes section to use https://github.com/open-horizon/anax/releases/latest/download/agent-install.sh when the necessary version of agent-install.sh is released to https://github.com/open-horizon/anax/releases
• Expose the agbot secure APIs (even before adding TLS support). For example:

    "SecureAPIListenHost": "0.0.0.0",
    "SecureAPIListenPort": "8083",

Occasionally, the ubuntu apt index is unavailable as they release an update. This causes updating the apt package index operation to fail, causing the apt-get update operation to return -1, which in turn bails on the deploy-mgmt-hub.sh script. Should this cause the script to fail, or should it be allowed to continue if the operator is notified and approves of continuing?

root@ubun2:~# curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash
----------- Verifying input and the host OS...
Manaagement hub services will listen on 127.0.0.1
Updating apt package index...
Error running apt-get update -q: Hit:1 http://us.archive.ubuntu.com/ubuntu bionic InRelease
Get:2 http://us.archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Reading package lists...
E: Release file for http://us.archive.ubuntu.com/ubuntu/dists/bionic-updates/InRelease is not valid yet (invalid for another 1h 41min 20s). Updates for this repository will not be applied.
E: Release file for http://us.archive.ubuntu.com/ubuntu/dists/bionic-backports/InRelease is not valid yet (invalid for another 1h 42min 54s). Updates for this repository will not be applied.
E: Release file for http://us.archive.ubuntu.com/ubuntu/dists/bionic-security/InRelease is not valid yet (invalid for another 1h 40min 7s). Updates for this repository will not be applied.

I have installed a local all in one env, what 's password means? or where to use it

hzn exchange user list -v
[verbose] Reading configuration file: /usr/local/etc/horizon/hzn.json
[verbose] Reading configuration file: /etc/default/horizon
[verbose] Config file does not exist: /var/root/.hzn/hzn.json.
[verbose] No project level configuration file found.
[verbose] The exchange url: http://host.docker.internal:3090/v1
[verbose] GET http://host.docker.internal:3090/v1/orgs/myorg/users/admin
[verbose] HTTP request timeout set to 30 seconds
[verbose] HTTP code: 200
{
"myorg/admin": {
"admin": true,
"email": "not@used",
"hubAdmin": false,
"lastUpdated": "2020-11-30T08:07:14.655237Z[UTC]",
"password": "********",
"updatedBy": "root/root"
}
}

I wanted to deploy the horizon management hub on my macbook (OS = big sur)
For that I followed the instructions of https://github.com/open-horizon/devops/blob/master/mgmt-hub/README.md

I also set the HZN_LISTEN_IP before launching script.
See output here below.

Jans-MBP:~ root# export HZN_LISTEN_IP=192.168.1.5
Jans-MBP:~ root# echo $HZN_LISTEN_IP
192.168.1.5
Jans-MBP:~ root# curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash
Input error
Input error
Input error
Input error
Input error
----------- Verifying input and the host OS...
Error: these environment variables must be set: EXCHANGE_ROOT_PW, EXCHANGE_ROOT_PW_BCRYPTED
Jans-MBP:~ root# whoami
root
Jans-MBP:~ root# 

As you can see it is 5 times reporting Input error and it is reporting error:

Error: these environment variables must be set: EXCHANGE_ROOT_PW, EXCHANGE_ROOT_PW_BCRYPTED

Got the following errors in anax git hub Actions which invokes the all-in-in management hub.

------- Creating a Vault instance and preforming all setup and configuration operations ...
Checking Vault service status, initialization, and seal...
A Vault instance has not been initialized. Initializing...
Vault instance is sealed. Unsealing...
Creating KV ver.2 secrets engine openhorizon...
Registering auth plugin openhorizon-exchange to Vault instance...
Generating SHA256 hash of openhorizon-exchange plugin...
the input device is not a TTY
Error: http code {"errors":["missing from: value"]}
All-in-one management hub deployment: Failed deploy.

root# curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash

----------- Verifying input and the host OS...
Management hub services will listen on 127.0.0.1
Installing prerequisites using brew, this could take a minute...
Error running brew install jq gettext: Error: Running Homebrew as root is extremely dangerous and no longer supported.
As Homebrew does not drop privileges on installation you would be giving all
build scripts full access to your system.

MacOS: Big Sur 11.0.1

root@ip-10-0-0-122:# curl -sSL https://raw.githubusercontent.com/open-horizon/devops/master/mgmt-hub/deploy-mgmt-hub.sh | bash
----------- Verifying input and the host OS...
Error: the host must be Ubuntu 18.x (amd64, ppc64le) or Ubuntu 20.x (amd64, ppc64le) or macOS or RedHat 7.9 (ppc64le) or RedHat 8.3 (ppc64le)
root@ip-10-0-0-122:#
root@ip-10-0-0-122:#
root@ip-10-0-0-122:#
root@ip-10-0-0-122:# cat /etc/os-release
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
root@ip-10-0-0-122:#
root@ip-10-0-0-122:# DISTRO=${DISTRO:-$(. /etc/os-release 2>/dev/null;echo $ID $VERSION_ID)}
root@ip-10-0-0-122:#
root@ip-10-0-0-122:#
root@ip-10-0-0-122:# echo $DISTRO
ubuntu 20.04
root@ip-10-0-0-122:#
root@ip-10-0-0-122:#

We need GitHub Actions to build artifacts for OpenHorizon repositories and deploy those artifacts to either Dockerhub or GitHub Container Registry or Github Package Repository.

Each OpenHorizon repository would create Docker images and tag them testing and deploy. In addition, the Anax repository would need to build RPM and Debian packages and deploy them to the GitHub Package Repository.