iby / docker-riak-cs Goto Github PK

Firstly, thank you for perfect docker image.

I would like to store bigger data and make them persistent (docker service restart / server restart).
I guess that I can set custom --volume param, but unfortunately I do not know which directory I have to mount.

UPDATE
I can change storage by -v /my/storage:/var/lib/riak but unfortunately (and obviously) it doesn't work - this corrupt/clear the riak DB internal storage. Is it possible to fix it somehow?

I'm following the directions to set up/configure and am not able to connect.

I've downloaded a fresh Docker image from DockerHub. Run it. Seems healthy. If I climb inside with docker exec, I can run 'riak ping', 'riak-cs ping' and 'stanchion ping' and all return 'pong'.

I downloaded a fresh install of s3cmd (2.0.2) and configured with this .s3cfg file: (the 2 security keys came from the log output of the running container)

[default]
access_key = OPSBRJYKYIDJQMCDJU6Y
host_base = s3.amazonaws.dev
host_bucket = %(bucket)s.s3.amazonaws.dev
proxy_host = localhost
proxy_port = 8080
secret_key = eUARz8q1KDfap_Dp2t3solbr_8soo5Ubw8UaJA==
signature_v2 = True

The sample docker launch command given in the instructions here create an initial bucket, testbucket, so I thought I should be able to see it.

Running "s3cmd ls" I get this response:

WARNING: Retrying failed request: / (No status line received - the server has closed the connection)
WARNING: Waiting 3 sec...
WARNING: Retrying failed request: / (No status line received - the server has closed the connection)
WARNING: Waiting 6 sec...
^CSee ya!

Am I doing something wrong, or are the instructions not complete?
Thanks,
Greg

I did the following:

docker pull 'ianbytchek/riak-cs'
docker run -e 'RIAK_CS_BUCKETS=foo,bar,baz' --name 'riak-cs' ianbytchek/riak-cs

The output I am getting:

Starting Riak… OK!
Starting Stanchion… OK!
Starting Riak CS… OK!

############################################################

    Riak admin credentials, make note of them, otherwise you
    will not be able to access your files and data. Riak
    services will be restarted to take effect.

       Key: 
    Secret: 

############################################################

Restarting Riak… OK!
Restarting Stanchion… OK!
Restarting Riak CS… OK!
Creating Riak CS buckets.
… Failed!
… Failed!
… Failed!
2015-09-20 09:18:10,970 CRIT Supervisor running as root (no user in config file)
2015-09-20 09:18:10,970 WARN Included extra file "/docker/configuration/supervisord/riak.conf" during parsing
2015-09-20 09:18:10,970 WARN Included extra file "/docker/configuration/supervisord/riak-cs.conf" during parsing
2015-09-20 09:18:10,970 WARN Included extra file "/docker/configuration/supervisord/stanchion.conf" during parsing
2015-09-20 09:18:10,973 INFO supervisord started with pid 3511
2015-09-20 09:18:11,975 INFO spawned: 'riak' with pid 3514
2015-09-20 09:18:11,976 INFO spawned: 'stanchion' with pid 3515
2015-09-20 09:18:11,978 INFO spawned: 'riak-cs' with pid 3516
2015-09-20 09:18:12,981 INFO success: riak entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-09-20 09:18:12,981 INFO success: stanchion entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2015-09-20 09:18:12,981 INFO success: riak-cs entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

It doesn't output credentials, I am unable to connect with it.

Hi,

I am unable to get the image to work. It fails to create the initial buckets and rejects any commands. Here is the logs, maybe something that can be fixed:

016-03-25 01:48:45.157 [error] <0.285.0> Supervisor {<0.285.0>,poolboy_sup} had child riak_cs_riakc_pool_worker started with riak_cs_riakc_pool_worker:start_link([{address,"127.0.0.1"},{port,8087}]) at undefined exit with reason shutdown in context shutdown_error
2016-03-25 01:48:45.160 [error] <0.278.0> Supervisor {<0.278.0>,poolboy_sup} had child riak_cs_riak_client started with riak_cs_riak_client:start_link([]) at undefined exit with reason shutdown in context shutdown_error
2016-03-25 01:48:45.161 [error] <0.148.0> Supervisor {<0.148.0>,poolboy_sup} had child riak_cs_riak_client started with riak_cs_riak_client:start_link([]) at undefined exit with reason shutdown in context shutdown_error
2016-03-25 01:48:49.342 [error] <0.431.0>@riak_cs_wm_common:maybe_create_user:210 Retrieval of user record for s3 failed. Reason: <<"{pr_val_unsatisfied,1,0}">>
2016-03-25 01:48:49.374 [error] <0.432.0>@riak_cs_wm_common:maybe_create_user:210 Retrieval of user record for s3 failed. Reason: <<"{pr_val_unsatisfied,1,0}">>
2016-03-25 01:48:49.385 [error] <0.433.0>@riak_cs_wm_common:maybe_create_user:210 Retrieval of user record for s3 failed. Reason: <<"{pr_val_unsatisfied,1,0}">>
2016-03-25 01:48:49.396 [error] <0.434.0>@riak_cs_wm_common:maybe_create_user:210 Retrieval of user record for s3 failed. Reason: <<"{pr_val_unsatisfied,1,0}">>
2016-03-25 01:48:49.406 [error] <0.435.0>@riak_cs_wm_common:maybe_create_user:210 Retrieval of user record for s3 failed. Reason: <<"{pr_val_unsatisfied,1,0}">>
2016-03-25 01:48:49.418 [error] <0.436.0>@riak_cs_wm_common:maybe_create_user:210 Retrieval of user record for s3 failed. Reason: <<"{pr_val_unsatisfied,1,0}">>
2016-03-25 02:06:08.100 [error] <0.3911.0> CRASH REPORT Process <0.3911.0> with 0 neighbours crashed with reason: no function clause matching riak_cs_s3_rewrite:separate_bucket_from_path("s3.amazonaws.dev:443") line 173

We would like to use your container to provide our developers environment (based on a set of docker images) with a local s3, we're using a Vagrantfile to combine the different containers and to do the linking/mounting/provisionning

The problem we're having with your containers is that we can not create a user with read/write access on a bucket (as you state in the readme, if we set the RIAK_CS_KEY_ACCESS and RIAK_CS_KEY_SECRET it fails )

it would be nice to either

• output the admin credentials that are currently output in stdout, in a file, e.g /var/whatever/credentials , so that we can mount /var/whatever from other containers
  • have environment variables that if set , create a non-admin user that have the read/write rights on the created buckets

Any chance you would be willing to add a license to the code in this repository?

I'm looking to simulate S3 locally. On debian stretch amd64 (a Thinkpad X1 Gen 6 Carbon), it takes about 98s from typing

docker run -p 127.0.0.1:1234:8080  ianbytchek/riak-cs

till the final output

2018-09-10 11:37:15,595 INFO success: riak-cs entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Until that final output, riak is not usable, and the CPU load is 100%.

In contrast, competing projects take about 0.1s to start.

If there is any compilation needed, it should be done as part of the build process, not during spin-up of the container.

==========docker-compose.yml=============
riak-cs:
image: ianbytchek/riak-cs
container_name: riak-cs-amazon-s3
healthcheck:
timeout: 60s
retries: 10
environment:
- RIAK_CS_BUCKETS=riak
- RIAK_CS_KEY_ACCESS=CC2QOJQ4VJFR8USBS5Y3
- RIAK_CS_KEY_SECRET=dKoX0b8dKbSFS0lJ_euHAVdgHMPDuB9GDZgJ8Q==
ports:
- "8080:8080"

========docker logs===========
riak-cs-amazon-s3 | Waiting for riak kv service to startup… OK!
riak-cs-amazon-s3 | Starting Stanchion… OK!
riak-cs-amazon-s3 | Starting Riak CS… OK!
riak-cs-amazon-s3 |
riak-cs-amazon-s3 | ############################################################
riak-cs-amazon-s3 |
riak-cs-amazon-s3 | Riak admin credentials, make note of them, otherwise you
riak-cs-amazon-s3 | will not be able to access your files and data. Riak
riak-cs-amazon-s3 | services will be restarted to take effect.
riak-cs-amazon-s3 |
riak-cs-amazon-s3 | Access key: CC2QOJQ4VJFR8USBS5Y3
riak-cs-amazon-s3 | Secret key: dKoX0b8dKbSFS0lJ_euHAVdgHMPDuB9GDZgJ8Q==
riak-cs-amazon-s3 |
riak-cs-amazon-s3 | ############################################################
riak-cs-amazon-s3 |
riak-cs-amazon-s3 | Restarting Riak… OK!
riak-cs-amazon-s3 | Waiting for riak kv service to startup… OK!
riak-cs-amazon-s3 | Restarting Stanchion…Restarting Riak CS…Node '[email protected]' not responding to pings.
riak-cs-amazon-s3 | Node '[email protected]' not responding to pings.
riak-cs-amazon-s3 | OK!
riak-cs-amazon-s3 | Creating Riak CS buckets.
riak-cs-amazon-s3 | riak… Failed!
riak-cs-amazon-s3 | 2018-02-19 09:35:48,371 CRIT Supervisor running as root (no user in config file)
riak-cs-amazon-s3 | 2018-02-19 09:35:48,371 WARN Included extra file "/entrypoint/configuration/supervisord/stanchion.conf" during parsing
riak-cs-amazon-s3 | 2018-02-19 09:35:48,371 WARN Included extra file "/entrypoint/configuration/supervisord/riak-cs.conf" during parsing
riak-cs-amazon-s3 | 2018-02-19 09:35:48,371 WARN Included extra file "/entrypoint/configuration/supervisord/riak.conf" during parsing
riak-cs-amazon-s3 | 2018-02-19 09:35:48,374 INFO supervisord started with pid 4906
riak-cs-amazon-s3 | 2018-02-19 09:35:49,376 INFO spawned: 'riak' with pid 4909
riak-cs-amazon-s3 | 2018-02-19 09:35:49,378 INFO spawned: 'stanchion' with pid 4910
riak-cs-amazon-s3 | 2018-02-19 09:35:49,380 INFO spawned: 'riak-cs' with pid 4911
riak-cs-amazon-s3 | 2018-02-19 09:35:49,839 INFO exited: riak-cs (exit status 1; not expected)
riak-cs-amazon-s3 | 2018-02-19 09:35:49,859 INFO exited: stanchion (exit status 1; not expected)
riak-cs-amazon-s3 | 2018-02-19 09:35:51,001 INFO success: riak entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
riak-cs-amazon-s3 | 2018-02-19 09:35:51,007 INFO spawned: 'stanchion' with pid 5248
riak-cs-amazon-s3 | 2018-02-19 09:35:51,013 INFO spawned: 'riak-cs' with pid 5249
riak-cs-amazon-s3 | 2018-02-19 09:35:51,440 INFO exited: stanchion (exit status 1; not expected)
riak-cs-amazon-s3 | 2018-02-19 09:35:51,492 INFO exited: riak-cs (exit status 1; not expected)
riak-cs-amazon-s3 | 2018-02-19 09:35:53,494 INFO spawned: 'stanchion' with pid 5454
riak-cs-amazon-s3 | 2018-02-19 09:35:53,496 INFO spawned: 'riak-cs' with pid 5455
riak-cs-amazon-s3 | 2018-02-19 09:35:53,860 INFO exited: stanchion (exit status 1; not expected)
riak-cs-amazon-s3 | 2018-02-19 09:35:53,874 INFO exited: riak-cs (exit status 1; not expected)
riak-cs-amazon-s3 | 2018-02-19 09:35:56,878 INFO spawned: 'stanchion' with pid 5660
riak-cs-amazon-s3 | 2018-02-19 09:35:56,879 INFO spawned: 'riak-cs' with pid 5661
riak-cs-amazon-s3 | 2018-02-19 09:35:57,254 INFO exited: stanchion (exit status 1; not expected)
riak-cs-amazon-s3 | 2018-02-19 09:35:57,256 INFO gave up: stanchion entered FATAL state, too many start retries too quickly
riak-cs-amazon-s3 | 2018-02-19 09:35:57,256 INFO exited: riak-cs (exit status 1; not expected)
riak-cs-amazon-s3 | 2018-02-19 09:35:58,257 INFO gave up: riak-cs entered FATAL state, too many start retries too quickly

Admin must be created only if it doesn't exist, same applies to buckets.

Should be on default port 8098, no luck to find correct config parameter to listen on ip not loopback (versions?) http://docs.basho.com/riak/kv/2.2.0/using/admin/riak-control/

Invoked as: /usr/local/bin/s3cmd ls s3://foo
Problem: BadStatusLine: ''
S3cmd: 1.6.1
python: 2.7.11 (default, Jan 22 2016, 08:29:18)
[GCC 4.2.1 Compatible Apple LLVM 7.0.2 (clang-700.1.81)]
environment LANG=None

Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/s3cmd-1.6.1-py2.7.egg/EGG-INFO/scripts/s3cmd", line 2996, in
File "/usr/local/lib/python2.7/site-packages/s3cmd-1.6.1-py2.7.egg/EGG-INFO/scripts/s3cmd", line 2915, in main
File "/usr/local/lib/python2.7/site-packages/s3cmd-1.6.1-py2.7.egg/EGG-INFO/scripts/s3cmd", line 137, in cmd_ls
File "/usr/local/lib/python2.7/site-packages/s3cmd-1.6.1-py2.7.egg/EGG-INFO/scripts/s3cmd", line 170, in subcmd_bucket_list
File "build/bdist.macosx-10.11-x86_64/egg/S3/S3.py", line 327, in bucket_list
for truncated, dirs, objects in self.bucket_list_streaming(bucket, prefix, recursive, uri_params, limit):
File "build/bdist.macosx-10.11-x86_64/egg/S3/S3.py", line 358, in bucket_list_streaming
response = self.bucket_list_noparse(bucket, prefix, recursive, uri_params, max_keys)
File "build/bdist.macosx-10.11-x86_64/egg/S3/S3.py", line 389, in bucket_list_noparse
response = self.send_request(request)
File "build/bdist.macosx-10.11-x86_64/egg/S3/S3.py", line 1235, in send_request
conn = ConnMan.get(self.get_hostname(resource['bucket']))
File "build/bdist.macosx-10.11-x86_64/egg/S3/ConnMan.py", line 249, in get
conn.c.connect()
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1270, in connect
HTTPConnection.connect(self)
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 839, in connect
self._tunnel()
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 811, in _tunnel
(version, code, message) = response._read_status()
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 417, in _read_status
raise BadStatusLine(line)
BadStatusLine: ''