We should run stellar-core --newhist <historyarchive> for all history archives that have a put command.

Thanks to @ramontayag for raising the issue in the Stellar slack.

New upstream release: https://github.com/stellar/stellar-core/releases/tag/v9.0.0

Stellar Core 9.1.0 RC (RC1?) has been released: https://github.com/stellar/stellar-core/releases/tag/v9.1.0.

Basically, allow a user to specify an s3 history option similar to this:
https://github.com/stellar/stellar-core/blob/master/docs/stellar-core_example.cfg#L314

Currently the config is written to a file in /. This requires

1. That the container runs as root
2. That the container's filesystem is writable

See https://github.com/satoshipay/docker-stellar-core/blob/master/confd/conf.d/stellar-core.cfg.toml#L3

This should be changed to any other subdirectory, such as /config so that the user of this image can make the container filesystem read only and/or control the user that it runs under.

I cloned this repo and tried to run the functional node to run a Prometheus exporter against it, but I am having this issue. Also I tried setting the environment variable in the dockerfile but that doesn't seem to work either. Where do I specify the password? the DB should already be initializing so the only issue should be a non-specified password right?

kentarovadney@Kentaros-MacBook-Air docker-stellar-core % docker-compose -f docker-compose.example.yml up
Docker Compose is now in the Docker CLI, try docker compose up

Creating docker-stellar-core_stellar-core-postgres_1 ... done
Creating docker-stellar-core_stellar-core_1 ... done
Attaching to docker-stellar-core_stellar-core-postgres_1, docker-stellar-core_stellar-core_1
stellar-core-postgres_1 | Error: Database is uninitialized and superuser password is not specified.
stellar-core-postgres_1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
stellar-core-postgres_1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
stellar-core-postgres_1 |
stellar-core-postgres_1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
stellar-core-postgres_1 | connections without a password. This is not recommended.
stellar-core-postgres_1 |
stellar-core-postgres_1 | See PostgreSQL documentation about "trust":
stellar-core-postgres_1 | https://www.postgresql.org/docs/current/auth-trust.html
stellar-core_1 | Initializing core db...
stellar-core_1 | Using DEPRECATED command-line syntax.
stellar-core_1 | Please refer to documentation for new syntax.
stellar-core_1 |
stellar-core_1 | 2021-06-26T22:34:33.027 [default INFO] Assigning calculated value of 1 to FAILURE_SAFETY
stellar-core_1 | 2021-06-26T22:34:33.061 GD4E3 [Database INFO] Connecting to: postgresql://dbname=stellar-core user=postgres password=******** host=stellar-core-postgres
docker-stellar-core_stellar-core-postgres_1 exited with code 1
stellar-core_1 | 2021-06-26T22:34:36.179 GD4E3 [default INFO] Application destructing
stellar-core_1 | 2021-06-26T22:34:36.186 GD4E3 [default INFO] Application destroyed
stellar-core_1 | 2021-06-26T22:34:36.188 GD4E3 [default FATAL] Got an exception: Cannot establish connection to the database.
stellar-core_1 | could not connect to server: No route to host
stellar-core_1 | Is the server running on host "stellar-core-postgres" (172.19.0.2) and accepting
stellar-core_1 | TCP/IP connections on port 5432?
stellar-core_1 | [DeprecatedCommandLine.cpp:444]
stellar-core-postgres_1 | Error: Database is uninitialized and superuser password is not specified.
stellar-core-postgres_1 | You must specify POSTGRES_PASSWORD to a non-empty value for the
stellar-core-postgres_1 | superuser. For example, "-e POSTGRES_PASSWORD=password" on "docker run".
stellar-core-postgres_1 |
stellar-core-postgres_1 | You may also use "POSTGRES_HOST_AUTH_METHOD=trust" to allow all
stellar-core-postgres_1 | connections without a password. This is not recommended.
stellar-core-postgres_1 |
stellar-core-postgres_1 | See PostgreSQL documentation about "trust":
stellar-core-postgres_1 | https://www.postgresql.org/docs/current/auth-trust.html
kentarovadney@Kentaros-MacBook-Air docker-stellar-core % docker-compose -f docker-compose.example.yml up -e POSTGRES_PASSWORD=password

There are a number of issues in the code but they can be easily rectified.

Need to add the following to the environment section in yaml

environment: (recommended)
- POSTGRES_DB=stellar-core
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD="P@ssw0rd" # Of course this is not the password, lol

However, when I ran the gen-seed command, it worked, but it seems I cannot connect to any of the peers:
docker run --rm -it --entrypoint '' satoshipay/stellar-core stellar-core gen-seed
Secret seed: SABBNWGOQ6KDNMOOKJA67FDYB2C3SFR4Q4FLOEQC2WGOUN37E2E2ODOI (sample)
Public: GACLTVRCSWNSJTBJO3GH3IF5LR5NCZCNTLS4CEL6FRFKGNVDOOCGSLPZ (sample)

In addition, the public key has spaces in it as well, of course that was commented out

NODE_SEED=SABBNWGOQ6KDNMOOKJA67FDYB2C3SFR4Q4FLOEQC2WGOUN37E2E2ODOI

That does not seem to work.

Also, I set the NETWORK_PASSPHRASE to

• NETWORK_PASSPHRASE='Test SDF Network ; September 2015'

Currently this is what it looks like

example docker compose file for how to run stellar core

version: '3'
services:
stellar-core-postgres:
image: postgres:9
restart: unless-stopped
environment:
- POSTGRES_DB=stellar-core
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD="P@ssw0rd"
ports:
- 5432:5432

stellar-core:
image: satoshipay/stellar-core:10.2.0
restart: unless-stopped
ports:
- 11625:11625
- 11626:11626
environment:
- DATABASE_PASSWORD="P@ssw0rd"
- DATABASE=postgresql://dbname=stellar-core user=postgres password=DATABASE_PASSWORD host=stellar-core-postgres
# WARNING: make sure to use a new NODE_SEED!
#- public_key=GBOXFAW5S4UUREXUQYO5EUT5UGGR5MLUKRMJM3HTJ2YFKUPWSLKDXEWV
#- PUBLIC_KEY=GBBLTVRCSWNSJTBJO3GH3IF5LR5NCZCNTLS4CEL6FRFKGNVDOOCGSLPZ
#- NODE_SEED=GBBLTVRCSWNSJTBJO3GH3IF5LR5NCZCNTLS4CEL6FRFKGNVDOOCGSLPZ my_stellar_node
#- NODE_SEED=SDEJ55PQ4FZBOMJB2X77ROJZUGAE2PMF7M42G5RKBZXQJEHLHNYDSQ47 atl_stellar_node
- NODE_SEED=SACKNWGOQ6KDNMOOKJA67FDYB2C3SFR4Q4FLOEQC2WGOUN37E2E2ODOI atl_crypto_node
- NODE_IS_VALIDATOR=true
- NETWORK_PASSPHRASE='Test SDF Network ; September 2015'
#- NETWORK_PASSPHRASE=Public Global Stellar Network ; September 2015
- INITIALIZE_DB=true
- CATCHUP_RECENT=60480
- "KNOWN_PEERS=
#core-live-a.stellar.org:11625,
#core-live-b.stellar.org:11625,
#core-live-c.stellar.org:11625,
au.stellar.ibm.com,
br.stellar.ibm.com,
ca.stellar.ibm.com,
no.stellar.ibm.com,
in.stellar.ibm.com,
uk.stellar.ibm.com,
us.stellar.ibm.com,
stellar-de-fra.satoshipay.io,
stellar-sg-sin.satoshipay.io,
stellar-us-iowa.satoshipay.io,
core-live-a.stellar.org,
core-live-b.stellar.org,
core-live-c.stellar.org"
- "PREFERRED_PEERS=
au.stellar.ibm.com,
br.stellar.ibm.com,
ca.stellar.ibm.com,
no.stellar.ibm.com,
in.stellar.ibm.com,
uk.stellar.ibm.com,
us.stellar.ibm.com,
stellar-de-fra.satoshipay.io,
stellar-sg-sin.satoshipay.io,
stellar-us-iowa.satoshipay.io,
core-live-a.stellar.org,
core-live-b.stellar.org,
core-live-c.stellar.org"
- "NODE_NAMES=
GCKWUQGSVO45ZV3QK7POYL7HMFWDKWJVMFVEGUJKCAEVUITUCTQWFSM6 ibm_au,
GBUJA3Z5TLAKLI5MEH4TETLXJBQVSVW74MNEKP5UUHTP3IMLNSUPOTVA ibm_br,
GB2HF2NHRKKFZYFDGD7MUENOYROOEK7SWYV2APYOODP6P7BUJTLILKIL ibm_ca,
GDRA72H7JWXAXWJKOONQOPH3JKNSH5MQ6BO5K74C3X6FO2G3OG464BPU ibm_no,
GCH3O5PTCZVR4G65W3B4XDKWI5V677HQB3QO7CW4YPVYDDFBE2GE7G6V ibm_in,
GAENPO2XRTTMAJXDWM3E3GAALNLG4HVMKJ4QF525TR25RI42YPEDULOW ibm_uk,
GARBCBH4YSHUJLYEPKEPMVYZIJ3ZSQR3QCJ245CWGY64X72JLN4A6RSG ibm_us,
GC5SXLNAM3C4NMGK2PXK4R34B5GNZ47FYQ24ZIBFDFOCU6D4KBN4POAE satoshipay-de-fra,
GBJQUIXUO4XSNPAUT6ODLZUJRV2NPXYASKUBY4G5MYP3M47PCVI55MNT satoshipay-sg-sin,
GAK6Z5UVGUVSEK6PEOCAYJISTT5EJBB34PN3NOLEQG2SUKXRVV2F6HZY satoshipay-us-iowa,
GCGB2S2KGYARPVIA37HYZXVRM2YZUEXA6S33ZU5BUDC6THSB62LZSTYH sdf_watcher1,
GCM6QMP3DLRPTAZW2UZPCPX2LF3SXWXKPMP3GKFZBDSF3QZGV2G5QSTK sdf_watcher2,
GABMKJM6I25XI4K7U6XWMULOUQIQ27BCTMLS6BYYSOWKTBUXVRJSXHYQ sdf_watcher3"
# this is just an example quorum set – you most likely want to change it
- >-
QUORUM_SET=[
{
"threshold_percent": 66,
"validators": [
"$$satoshipay-de-fra",
"$$satoshipay-sg-sin",
"$$satoshipay-us-iowa"
]
},
{
"path": "ibm",
"threshold_percent": 51,
"validators": [
"$$ibm_au",
"$$ibm_br",
"$$ibm_ca",
"$$ibm_no",
"$$ibm_in",
"$$ibm_uk",
"$$ibm_us"
]
},
{
"path": "sdf",
"threshold_percent": 51,
"validators": [
"$$sdf_watcher1",
"$$sdf_watcher2",
"$$sdf_watcher3"
]
}
]
- >-
HISTORY={
"satoshipay-de-fra": {"get": "curl -sf https://stellar-history-de-fra.satoshipay.io/{0} -o {1}"},
"satoshipay-sg-sin": {"get": "curl -sf https://stellar-history-sg-sin.satoshipay.io/{0} -o {1}"},
"satoshipay-us-iowa": {"get": "curl -sf https://stellar-history-us-iowa.satoshipay.io/{0} -o {1}"},
"sdf1": {"get": "curl -sf http://history.stellar.org/prd/core-live/core_live_001/{0} -o {1}"},
"sdf2": {"get": "curl -sf http://history.stellar.org/prd/core-live/core_live_002/{0} -o {1}"},
"sdf3": {"get": "curl -sf http://history.stellar.org/prd/core-live/core_live_003/{0} -o {1}"}
}

For test/dev enviroments.
Is it possible to add ability for ready.sh script to optionally --forcescp if state = 'Joining SCP'. This would help when running in K8s but on a single node network in dev/test.

If they bloat the image too much we should think about creating additional flavors for s3 and gcloud. They should be built automatically whenever we update the base image.

This was an issue before, see #1 and #2.

See https://github.com/bloom-solutions/docker-stellar-core/blob/master/scripts/entrypoint.sh#L5.

Thanks @ramontayag for the suggestion.

My setup might be different, but I was having sync issues until I ran stellar-core --conf /stellar-core.cfg --newdb. I see that this repo has the command in entry.sh (except it's not double-hyphen -- not sure if it makes a difference), and seems if I had run entry.sh before stellar-core --conf /stellar-core.cfg --newdb, I would not have seen those sync issues.

I don't mind adding the instructions, but wanted to bring it up because maybe it's my setup that's weird or there's a dockerism I'm not following.

FYI the latest stellar core release has been out for 4 days: https://github.com/stellar/stellar-core/releases/tag/v11.4.0