apache / rocketmq-docker Goto Github PK
View Code? Open in Web Editor NEWApache RocketMQ Docker
Home Page: https://rocketmq.apache.org/
License: Apache License 2.0
Apache RocketMQ Docker
Home Page: https://rocketmq.apache.org/
License: Apache License 2.0
https://archive.apache.org/dist/rocketmq/4.9.2/rocketmq-all-4.9.2-bin-release.zip no rocketmq-all*/* directory after unzip.
before version 4.9.2 is OK
rocketmq-docker/image-build/Dockerfile-alpine:
mv rocketmq-all*/* .
build rocketmq docker image with version 4.9.2:
sh build-image.sh 4.9.2 alpine
error "gpg: not a detached signature" occurred
macos big sur
build rocketmq docker image under version 4.9.1(<=4.9.1), it works fine.
RMQ/NameServer supports TLS/SSL configuration. So It is suggested to add a related usage case for docker
BUG REPORT
https://github.com/apache/rocketmq-docker/blob/master/image-build/build-image.sh
What did you do (The steps to reproduce)?
What did you expect to see?
What did you see instead?
Please tell us about your environment:
Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
FEATURE REQUEST
Please describe the feature you are requesting.
Provide any additional detail on your proposed use case for this feature.
Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:
FEATURE REQUEST
Add a documentation on how to build a local image after changing the source code, for test purpose. For example we would like to build an image locally tagged with dev: rocketmq:dev
I have tried this link: apache/rocketmq#4704 but it does not resolve my issue.
1.git clone https://github.com/apache/rocketmq-docker.git
2.sh build-image.sh 4.8.0 alpine
and it show:
[internal] load metadata for docker.io/library/openjdk:8-alpine:
failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed commit on ref "unknown-sha256:78467497febe9766548044687e8fd1afc6ba8116bd2cb3f9d0b9293a75c06703": "unknown-sha256:78467497febe9766548044687e8fd1afc6ba8116bd2cb3f9d0b9293a75c06703" failed size validation: 9070 != 8757: failed precondition`
my computer info:
1.MacBook Pro (13-inch, M1, 2020) macos Monterey 12.3
2.Docker version 20.10.7, build f0df350
Base image: alpine
In the runbroker-customize.sh, numactl --interleave=all pwd > /dev/null 2>&1 call the numactl, which does not included in the image. And the broker will not start successfully.
Additionally, I find that the alpine image does not have the numactl in its repository. The version of alpine is 3.9, however the lowest version required by numactl is 3.10.
numactl in alpine 3.9
numactl in alpine 3.10
I don't see the file stage.sh.
This docker-compose script from rocketmq-docker
When I run this script, The broker has crashing after nameserv startup
docker-compose.yml
version: '2'
services:
namesrv:
image: rocketmqinc/rocketmq:4.4.0
container_name: rmqnamesrv
ports:
- 9876:9876
volumes:
- ./data/namesrv/logs:/home/rocketmq/logs
- ./data/namesrv/store:/home/rocketmq/store
command: sh mqnamesrv
broker:
image: rocketmqinc/rocketmq:4.4.0
container_name: rmqbroker
ports:
- 10909:10909
- 10911:10911
- 10912:10912
volumes:
- ./data/broker/logs:/home/rocketmq/logs
- ./data/broker/store:/home/rocketmq/store
- ./data/broker/conf/broker.conf:/opt/rocketmq-4.4.0/conf/broker.conf
#command: sh mqbroker -n namesrv:9876
command: sh mqbroker -n namesrv:9876 -c ../conf/broker.conf
depends_on:
- namesrv
broker.conf
brokerClusterName = DefaultCluster
brokerName = broker-a
brokerId = 0
deleteWhen = 04
fileReservedTime = 48
brokerRole = ASYNC_MASTER
flushDiskType = ASYNC_FLUSH
#set `brokerIP1` if you want to set physical IP as broker IP.
namesrvAddr = 外网ip:9876
brokerIP1 = 外网ip
My server is Ubuntu 18.04
Is this a bug?
$ ./play-docker.sh alpine
Play RocketMQ docker image of tag 4.9.4-alpine
Starting RocketMQ nodes...
60e7aadff726f958b3206be609636db8179fcb660cfaa5c2dde9a98fa7417377
5198c96aefc5027a4d747f17c655c5397097c36997737e090c46c20037b08f07
docker: Error response from daemon: Cannot link to a non running container: /rmqnamesrv AS /rmqbroker/namesrv.
One host is nameserver+ Master.The other one is Nameserver + Slave.
Hello,I download rocketmq-docker today , and I just want to run the Cluster Mode. I don't modify any file,and just deploy by the Readme.md. Unfortunately,It ‘s failed
docker-compose.yml:
version: '2'
services:
#Service for nameserver
namesrv:
image: apacherocketmq/rocketmq:4.5.0
container_name: rmqnamesrv
ports:
- 9876:9876
volumes:
- ./data/namesrv/logs:/home/rocketmq/logs
command: sh mqnamesrv
#Service for broker
broker:
image: apacherocketmq/rocketmq:4.5.0
container_name: rmqbroker
links:
- namesrv
ports:
- 10909:10909
- 10911:10911
- 10912:10912
environment:
- NAMESRV_ADDR=namesrv:9876
volumes:
- ./data/broker/logs:/home/rocketmq/logs
- ./data/broker/store:/home/rocketmq/store
- ./data/broker/conf/broker.conf:/opt/rocketmq-4.5.0/conf/broker.conf
command: sh mqbroker -c /opt/rocketmq-4.5.0/conf/broker.conf
#Service for another broker -- broker1
broker1:
image: apacherocketmq/rocketmq:4.5.0
container_name: rmqbroker-b
links:
- namesrv
ports:
- 10929:10909
- 10931:10911
- 10932:10912
environment:
- NAMESRV_ADDR=namesrv:9876
volumes:
- ./data1/broker/logs:/home/rocketmq/logs
- ./data1/broker/store:/home/rocketmq/store
- ./data1/broker/conf/broker.conf:/opt/rocketmq-4.5.0/conf/broker.conf
command: sh mqbroker -c /opt/rocketmq-4.5.0/conf/broker.conf
I run ./play-docker-compose.sh
and run docker ps
can see :
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9d4576befd65 apacherocketmq/rocketmq:4.5.0 "sh mqbroker -c /opt…" About a minute ago Up About a minute 0.0.0.0:10909->10909/tcp, 9876/tcp, 0.0.0.0:10911-10912->10911-10912/tcp rmqbroker
f97e6f652907 apacherocketmq/rocketmq:4.5.0 "sh mqnamesrv" About a minute ago Up About a minute 10909/tcp, 0.0.0.0:9876->9876/tcp, 10911-10912/tcp rmqnamesrv
But when I run docker exec -it 9d4576befd65 ./mqadmin clusterList -n localhost:9876
I see this bug:
org.apache.rocketmq.tools.command.SubCommandException: ClusterListSubCommand command failed
at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.execute(ClusterListSubCommand.java:93)
at org.apache.rocketmq.tools.command.MQAdminStartup.main0(MQAdminStartup.java:135)
at org.apache.rocketmq.tools.command.MQAdminStartup.main(MQAdminStartup.java:86)
Caused by: org.apache.rocketmq.remoting.exception.RemotingConnectException: connect to <null> failed
at org.apache.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(NettyRemotingClient.java:392)
at org.apache.rocketmq.client.impl.MQClientAPIImpl.getBrokerClusterInfo(MQClientAPIImpl.java:1180)
at org.apache.rocketmq.tools.admin.DefaultMQAdminExtImpl.examineBrokerClusterInfo(DefaultMQAdminExtImpl.java:275)
at org.apache.rocketmq.tools.admin.DefaultMQAdminExt.examineBrokerClusterInfo(DefaultMQAdminExt.java:222)
at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.printClusterBaseInfo(ClusterListSubCommand.java:172)
at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.execute(ClusterListSubCommand.java:88)
... 2 more
It is worth mentioning that I exec docker run -it apacherocketmq/rocketmq:4.5.0 sh
and want to view the broker config ,so I run sh mqbroker -m
:
2020-03-19 17\:12\:15 INFO main - namesrvAddr=
2020-03-19 17\:12\:15 INFO main - brokerIP1=172.17.0.2
2020-03-19 17\:12\:15 INFO main - brokerName=46ff3293104e
2020-03-19 17\:12\:15 INFO main - brokerClusterName=DefaultCluster
2020-03-19 17\:12\:15 INFO main - brokerId=0
2020-03-19 17\:12\:15 INFO main - autoCreateTopicEnable=true
2020-03-19 17\:12\:15 INFO main - autoCreateSubscriptionGroup=true
2020-03-19 17\:12\:15 INFO main - msgTraceTopicName=RMQ_SYS_TRACE_TOPIC
2020-03-19 17\:12\:15 INFO main - traceTopicEnable=false
2020-03-19 17\:12\:15 INFO main - rejectTransactionMessage=false
2020-03-19 17\:12\:15 INFO main - fetchNamesrvAddrByAddressServer=false
2020-03-19 17\:12\:15 INFO main - transactionTimeOut=6000
2020-03-19 17\:12\:15 INFO main - transactionCheckMax=15
2020-03-19 17\:12\:15 INFO main - transactionCheckInterval=60000
2020-03-19 17\:12\:15 INFO main - aclEnable=false
2020-03-19 17\:12\:15 INFO main - storePathRootDir=/home/rocketmq/store
2020-03-19 17\:12\:15 INFO main - storePathCommitLog=/home/rocketmq/store/commitlog
2020-03-19 17\:12\:15 INFO main - flushIntervalCommitLog=500
2020-03-19 17\:12\:15 INFO main - commitIntervalCommitLog=200
2020-03-19 17\:12\:15 INFO main - flushCommitLogTimed=false
2020-03-19 17\:12\:15 INFO main - deleteWhen=04
2020-03-19 17\:12\:15 INFO main - fileReservedTime=72
2020-03-19 17\:12\:15 INFO main - maxTransferBytesOnMessageInMemory=262144
2020-03-19 17\:12\:15 INFO main - maxTransferCountOnMessageInMemory=32
2020-03-19 17\:12\:15 INFO main - maxTransferBytesOnMessageInDisk=65536
2020-03-19 17\:12\:15 INFO main - maxTransferCountOnMessageInDisk=8
2020-03-19 17\:12\:15 INFO main - accessMessageInMemoryMaxRatio=40
2020-03-19 17\:12\:15 INFO main - messageIndexEnable=true
2020-03-19 17\:12\:15 INFO main - messageIndexSafe=false
2020-03-19 17\:12\:15 INFO main - haMasterAddress=
2020-03-19 17\:12\:15 INFO main - brokerRole=ASYNC_MASTER
2020-03-19 17\:12\:15 INFO main - flushDiskType=ASYNC_FLUSH
2020-03-19 17\:12\:15 INFO main - cleanFileForciblyEnable=true
2020-03-19 17\:12\:15 INFO main - transientStorePoolEnable=false
above this result , I find the namesrvAddr is empty , So I think maybe the broker or broker cluster don't start well . So I can't verify the cluster well like ReadMe.md
BUG REPORT
I want to use docker compose to build a rocketmq cluster with 2 nameserver nodes, 3 master nodes and 3 slave nodes. When I configure the following, the broker node cannot be started normally
version: '3.5'
services:
namesrv1:
image: apache/rocketmq:4.9.3
container_name: namesrv1
ports:
- 19876:9876
command: sh mqnamesrv
namesrv2:
image: apache/rocketmq:4.9.3
container_name: namesrv2
ports:
- 29876:9876
command: sh mqnamesrv
broker-m-1:
image: apache/rocketmq:4.9.3
container_name: broker-m-1
links:
- namesrv1:namesrv1
- namesrv2:namesrv2
ports:
- 10909:10909
- 10911:10911
- 10912:10912
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "192.168.12.1:19876"
JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
volumes:
- ./rocketmq/broker-m-1/conf/broker-m-1.properties:/home/rocketmq/rocketmq-4.9.3/conf/broker-m-1.properties
command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.3/conf/broker-m-1.properties autoCreateTopicEnable=true &
broker-s-1:
image: apache/rocketmq:4.9.3
container_name: broker-s-1
links:
- namesrv1:namesrv1
- namesrv2:namesrv2
ports:
- 11909:10909
- 11911:11911
- 11912:10912
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "192.168.12.1:19876"
JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
volumes:
- ./rocketmq/broker-s-1/conf/broker-s-1.properties:/home/rocketmq/rocketmq-4.9.3/conf/broker-s-1.properties
command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.3/conf/broker-s-1.properties autoCreateTopicEnable=true &
broker-m-2:
image: apache/rocketmq:4.9.3
container_name: broker-m-2
links:
- namesrv1:namesrv1
- namesrv2:namesrv1
ports:
- 12909:10909
- 12911:12911
- 12912:10912
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "192.168.12.1:19876"
JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
volumes:
- ./rocketmq/broker-m-2/conf/broker-m-2.properties:/home/rocketmq/rocketmq-4.9.3/conf/broker-m-2.properties
command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.3/conf/broker-m-2.properties autoCreateTopicEnable=true &
broker-s-2:
image: apache/rocketmq:4.9.3
container_name: broker-s-2
links:
- namesrv1:namesrv1
- namesrv2:namesrv2
ports:
- 13909:10909
- 13911:13911
- 13912:10912
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "192.168.12.1:19876"
JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
volumes:
- ./rocketmq/broker-s-3/conf/broker-s-3.properties:/home/rocketmq/rocketmq-4.9.3/conf/broker-s-3.properties
command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.3/conf/broker-s-3.properties autoCreateTopicEnable=true &
broker-m-3:
image: apache/rocketmq:4.9.3
container_name: broker-m-3
links:
- namesrv1:namesrv1
- namesrv2:namesrv1
ports:
- 14909:10909
- 14911:12911
- 14912:10912
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "192.168.12.1:19876"
JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
volumes:
- ./rocketmq/broker-m-3/conf/broker-m-3.properties:/home/rocketmq/rocketmq-4.9.3/conf/broker-m-3.properties
command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.3/conf/broker-m-3.properties autoCreateTopicEnable=true &
broker-s-3:
image: apache/rocketmq:4.9.3
container_name: broker-s-3
links:
- namesrv1:namesrv1
- namesrv2:namesrv2
ports:
- 15909:10909
- 15911:13911
- 15912:10912
environment:
TZ: Asia/Shanghai
NAMESRV_ADDR: "192.168.12.1:19876"
JAVA_OPT_EXT: "-server -Xms256m -Xmx256m -Xmn256m"
volumes:
- ./rocketmq/broker-s-3/conf/broker-s-3.properties:/home/rocketmq/rocketmq-4.9.3/conf/broker-s-3.properties
command: sh mqbroker -c /home/rocketmq/rocketmq-4.9.3/conf/broker-s-3.properties autoCreateTopicEnable=true &
rocketmq-console:
image: styletang/rocketmq-console-ng
container_name: rocketmq-console
ports:
- 8090:8080
environment:
JAVA_OPTS: -Drocketmq.namesrv.addr=192.168.12.1:19876;192.168.12.1:29876 -Dcom.rocketmq.sendMessageWithVIPChannel=false
links:
- namesrv1:namesrv1
- namesrv2:namesrv2
#所属集群名称,如果多个master,那么每个master配置的名称应该一致,要不然识别不了
brokerClusterName=rocketmq-cluster
#broker名称
brokerName=broker-a
#0 表示master,>0 表示slave
brokerId=0
#nameServer地址,分号隔开
namesrvAddr=192.168.12.1:19876;192.168.12.1:29876
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许broker自动创建topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许broker自动创建订阅组,建议线下开始,线上关闭
autoCreateSubscriptionGroup=true
#broker对外服务的监听端口,
#同一台机器部署多个broker,端口号要不同,且端口号之间要相距大些
listenPort=10911
#删除文件的时间节点,默认凌晨4点
deleteWhen=04
#文件保留时间,默认48小时
fileReservedTime=120
#commitLog每个文件的大小,默认大小1g
mapedFileSizeCommitLog=1073741824
#consumeQueue每个文件默认存30w条,根据自身业务进行调整
mapedFileSizeConsumeQueue=300000
destroyMapedFileInterval=120000
redeleteHangedFileInterval=120000
#检查物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#store存储路径,master与slave目录要不同
storePathRootDir=./rocketmq/broker-m-1/data
#commitLog存储路径
# storePathCommitLog=/usr/local/rocketmq/rocketmq-4.2/data/store/commitlog
#限制的消息大小
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
#发消息线程池数
sendMessageThreadPoolNums=128
#拉去消息线程池数
pullMessageThreadPoolNums=128
#broker角色:
#ASYSC_MASTER 异步复制master
#SYSC_MASTER 同步复制master
#SLAVE 从
brokerRole=SYSC_MASTER
#刷盘方式
#ASYNC_FLUSH 异步刷盘
#SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
#所属集群名称,如果多个master,那么每个master配置的名称应该一致,要不然识别不了
brokerClusterName=rocketmq-cluster
#broker名称
brokerName=broker-a
#0 表示master,>0 表示slave
brokerId=1
#nameServer地址,分号隔开
namesrvAddr=192.168.12.1:19876;192.168.12.1:29876
#在发送消息时,自动创建服务器不存在的topic,默认创建的队列数
defaultTopicQueueNums=4
#是否允许broker自动创建topic,建议线下开启,线上关闭
autoCreateTopicEnable=true
#是否允许broker自动创建订阅组,建议线下开始,线上关闭
autoCreateSubscriptionGroup=true
#broker对外服务的监听端口,
#同一台机器部署多个broker,端口号要不同,且端口号之间要相距大些
listenPort=10911
#删除文件的时间节点,默认凌晨4点
deleteWhen=04
#文件保留时间,默认48小时
fileReservedTime=120
#commitLog每个文件的大小,默认大小1g
mapedFileSizeCommitLog=1073741824
#consumeQueue每个文件默认存30w条,根据自身业务进行调整
mapedFileSizeConsumeQueue=300000
destroyMapedFileInterval=120000
redeleteHangedFileInterval=120000
#检查物理文件磁盘空间
diskMaxUsedSpaceRatio=88
#store存储路径,master与slave目录要不同
storePathRootDir=./rocketmq/broker-s-1/data
#commitLog存储路径
# storePathCommitLog=/usr/local/rocketmq/rocketmq-4.2/data/store/commitlog
#限制的消息大小
maxMessageSize=65536
flushCommitLogLeastPages=4
flushConsumeQueueLeastPages=2
flushCommitLogThoroughInterval=10000
flushConsumeQueueThoroughInterval=60000
checkTransactionMessageEnable=false
#发消息线程池数
sendMessageThreadPoolNums=128
#拉去消息线程池数
pullMessageThreadPoolNums=128
#broker角色:
#ASYSC_MASTER 异步复制master
#SYSC_MASTER 同步复制master
#SLAVE 从
brokerRole=SYSC_MASTER
#刷盘方式
#ASYNC_FLUSH 异步刷盘
#SYNC_FLUSH 同步刷盘
flushDiskType=ASYNC_FLUSH
""Other node configurations are the same except for the three parameters of brokername brokerid storepathrootdir""
Cluster starts normally
IP : 192.168.12.1
Docker Desktop for Windows 11
In addition, I also tried to run on the Linux machine, but still failed to start normally
Because the broker is not started normally, I cannot see the log
add helm chart and dockerfile for CI
The docker image is still 4.6.0 https://hub.docker.com/r/apacherocketmq/rocketmq/tags
docker-compose.yml 中的配置文件是在 /home/rocketmq/ 不是在 /opt 下!!!
- ./data1/broker/conf/broker.conf:/opt/rocketmq-ROCKETMQ_VERSION/conf/broker.conf
[root@vbox1 image-build]# docker pull apacherocketmq/rocketmq
Using default tag: latest
Error response from daemon: manifest for apacherocketmq/rocketmq:latest not found: manifest unknown: manifest unknown
The issue tracker is ONLY used for bug report(feature request need to follow RIP process). Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our mailing lists. We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
BUG REPORT
ERROR: manifest for rocketmqinc/rocketmq:4.5.0 not found
Please tell us about your environment:
Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
FEATURE REQUEST
Please describe the feature you are requesting.
Provide any additional detail on your proposed use case for this feature.
Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:
i have found the note:
Actually the formal images have been generated by RocketMQ official maintainer and stored in docker hub.
but there is no tag like 'Official Image' like 'zookeeper'.
FROM amazon-corretto-8:latest
RUN apt-get update && apt-get install -y --no-install-recommends
bash libapr1 unzip telnet wget gnupg ca-certificates
&& rm -rf /var/lib/apt/lists/*
ARG user=rocketmq
ARG group=rocketmq
ARG uid=3000
ARG gid=3000
rocketmq
, uid = 3000RUN groupadd -g ${gid} ${group}
&& useradd -u ${uid} -g ${gid} -m -s /bin/bash ${user}
ARG version
ENV ROCKETMQ_VERSION ${version}
ENV ROCKETMQ_HOME /home/rocketmq/rocketmq-${ROCKETMQ_VERSION}
WORKDIR ${ROCKETMQ_HOME}
RUN set -eux;
&& wget https://archive.apache.org/dist/rocketmq/${ROCKETMQ_VERSION}/rocketmq-all-${ROCKETMQ_VERSION}-bin-release.zip -o rocketmq.zip;
&& wget https://archive.apache.org/dist/rocketmq/${ROCKETMQ_VERSION}/rocketmq-all-${ROCKETMQ_VERSION}-bin-release.zip.asc -o rocketmq.zip.asc;
&& #https://www.apache.org/dist/rocketmq/KEYS
&& wget https://www.apache.org/dist/rocketmq/KEYS -o KEYS;
&& gpg --import KEYS;
&& gpg --batch --verify rocketmq.zip.asc rocketmq.zip ;
&& unzip rocketmq.zip ;
&& mv rocketmq*/* . ;
&& rmdir rocketmq-* ;
&& rm rocketmq.zip rocketmq.zip.asc KEYS
is wrong
In,1.18.18,deployment.yaml is not work and I suggest to take a change like this:
1.apiVersion change to apps/v1
2.add the code
selector: matchLabels: app: rocketmq
in the field 'spec'
3.because of the openjdk,you need to add /usr/lib/jvm/jre-1.8.0-openjdk/lib/ext
in bin/tools.sh,like this:
JAVA_OPT="${JAVA_OPT} -server -Xms1g -Xmx1g -Xmn256m -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=128m" JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${BASE_DIR}/lib:${JAVA_HOME}/jre/lib/ext:${JAVA_HOME}/lib/ext:/usr/lib/jvm/jre-1.8.0-openjdk/lib/ext" JAVA_OPT="${JAVA_OPT} -cp ${CLASSPATH}"
Maybe it is not serious to the hacker,but it spent me some day to fix it.So please fix in the document or the code
rocketmq-docker/image-build/Dockerfile-centos
Line: 50-51
should be replaced by
BUG REPORT
The producer of the host send messages to the container cluster
Send success
Send fail
CentOS
FEATURE REQUEST
When users run RocketMQ by docker images, the startup scripts must be specified, e.g.:
docker run --name rmqnamesrv apache/rocketmq:4.9.2 sh mqnamesrv
It exposes too much details to users, for they should just need to specify which module (like nameserver, broker, etc.) is expected to be started, instead of the specific scripts.
I' like to add an entry point for RocketMQ docker image, making it more simple to launch specific modules, without the loss of flexibility.
An entry-point script will be added, which supports:
sh mqbroker
) as the current way.NODE_ROLE
.nameserver
broker
controller
The issue tracker is ONLY used for bug report(feature request need to follow RIP process). Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our mailing lists. We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
BUG REPORT
What did you do (The steps to reproduce)?
What did you expect to see?
What did you see instead?
Please tell us about your environment:
Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
FEATURE REQUEST
Please describe the feature you are requesting.
Provide any additional detail on your proposed use case for this feature.
Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:
The issue tracker is ONLY used for bug report(feature request need to follow RIP process). Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our mailing lists. We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
BUG REPORT
What did you do (The steps to reproduce)?
What did you expect to see?
What did you see instead?
Please tell us about your environment:
Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
FEATURE REQUEST
Please describe the feature you are requesting.
Provide any additional detail on your proposed use case for this feature.
Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:
docker run -d -p 9876:9876
-v /my/rocketmq/server/logs:/home/rocketmq/logs
-e TZ="Asia/Shanghai" -e "JAVA_OPT_EXT=-Xms1024M -Xmx1024M -Xmn128m"
--name rocketmq-server apache/rocketmq:latest
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.4 LTS
Release: 20.04
Codename: focal
run ./play-kubernetes.sh
get error:
error: unable to recognize "": no matches for kind "Deployment" in version "extensions/v1beta1"
使用下面的配置部署broker,启动报253错误
version: '3.5'
services:
rmqbroker-a:
image: apacherocketmq/rocketmq:4.5.0
container_name: rmqbroker-a
ports:
- 11909:10909
- 11911:10911
- 11912:10912
volumes:
- ./logs/broker-a:/home/rocketmq/logs
- ./store/broker-a:/home/rocketmq/store
- ./conf/broker-a/broker-a.conf:/home/rocketmq/rocketmq-4.5.0/conf/broker.conf
privileged: true
command: sh mqbroker -c ../conf/broker.conf autoCreateTopicEnable=true &
配置如下:
brokerClusterName = rocketmq-cluster
brokerName = broker-a
brokerId = 0
#brokerIP1 = 172.21.52.27
deleteWhen = 04
fileReservedTime = 48
namesrvAddr=172.21.52.27:9876
autoCreateTopicEnable=true
#Broker 对外服务的监听端口,
listenPort = 11911
#Broker角色
brokerRole=ASYNC_MASTER
#刷盘方式
flushDiskType=ASYNC_FLUSH
不知哪里出了问题
Adapt to the latest RocketMQ 5.0 version.
Where is 5.0?
the official docker image from
https://hub.docker.com/repository/docker/apacherocketmq/rocketmq
to
https://hub.docker.com/repository/docker/apache/rocketmq
FEATURE REQUEST
Add the list of supported RocketMQ versions.
in my company ,now want to use rocketmq.
then i think use 4.7.4
cd image-build,sh build-image.sh 4.7.4 centos
then:
[root@testecs-x image-build]# docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
1dbc85a3dd12 31 seconds ago 475MB
6bfac1be77b6 12 minutes ago 475MB
i do build two times.and rocketmq image is no REPOSITORY and no tag
sh build-image-dashboard.sh 1.0.0 centos build image is ok
089fbec3e3e0 rocketmqinc/rocketmq:4.4.0 "sh mqbroker -c /o..." 30 minutes ago Exited (253) 3 second ago
I fellow the 4 to generate rocketmq-dashboard. It runs in ubuntu20.04.2 docker env successfully, but I can't access dashboard.
by the way, my machine is Apple silicon.
I'm trying to deploy rocketmq on my testing cluster. I started from the scripts provided in that repo, but they do not work. I modified them in the following way:
apiVersion: apps/v1
kind: Deployment
metadata:
name: rocketmq-namesrv
spec:
replicas: 1
selector:
matchLabels:
app: rocketmq-namesrv
template:
metadata:
labels:
app: rocketmq-namesrv
spec:
containers:
- name: namesrv
image: myrepo/rocketmq:4.9.3-alpine
command: ["sh", "mqnamesrv"]
imagePullPolicy: IfNotPresent
resources:
limits:
memory: "128Mi"
cpu: "400m"
ports:
- containerPort: 9876
volumeMounts:
- name: namesrv-log
mountPath: /var/log
volumes:
- name: namesrv-log
persistentVolumeClaim:
claimName: rocketmq-namesrv-pvc
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: rocketmq-broker
spec:
replicas: 1
selector:
matchLabels:
app: rocketmq-broker
template:
metadata:
labels:
app: rocketmq-broker
spec:
containers:
- name: broker
image: myrepo/rocketmq:4.9.3-alpine
command: ["sh", "mqbroker", "-n", "localhost:9876"]
imagePullPolicy: IfNotPresent
resources:
limits:
memory: "128Mi"
cpu: "400m"
ports:
- containerPort: 10909
- containerPort: 10911
volumeMounts:
- name: broker-log
mountPath: /var/log
- name: broker-store
mountPath: /home/rocketmq
volumes:
- name: broker-log
persistentVolumeClaim:
claimName: rocketmq-broker-log-pvc
- name: broker-store
persistentVolumeClaim:
claimName: rocketmq-broker-store-pvc
The image rocketmq:4.9.3-alpine
was created following the procedure on that repo.
After the deployment the rocketmq-namesrv works, but the broker's pod logs: sh: can't open 'mqbroker': No such file or directory
...
What could it be? Am I making something wrong?
BUG REPORT
when I download zip from rocketmq, runserver.sh, runbroker.sh show that:
server : -Xms4g -Xmx4g -Xmn2g
broker: -Xms8g -Xmx8g -Xmn4g
but in runbroker-customize
desired_yg_in_mb=expr $max_heap_size_in_mb / 4
so what size is recommended for Xmn? 1/2 Xmx or 1/4 Xmx?
sh build-image.sh 4.4.0 alpine
Version = 4.4.0
Sending build context to Docker daemon 28.16 kB
Step 1/15 : FROM openjdk:8-alpine
---> a3562aa0b991
Step 2/15 : RUN apk add --no-cache bash gettext nmap-ncat openssl busybox-extras
---> Running in b4b349725631
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/15) Installing ncurses-terminfo-base (6.1_p20190105-r0)
(2/15) Installing ncurses-terminfo (6.1_p20190105-r0)
(3/15) Installing ncurses-libs (6.1_p20190105-r0)
(4/15) Installing readline (7.0.003-r1)
(5/15) Installing bash (4.4.19-r1)
Executing bash-4.4.19-r1.post-install
(6/15) Installing busybox-extras (1.29.3-r10)
Executing busybox-extras-1.29.3-r10.post-install
(7/15) Installing libgomp (8.3.0-r0)
(8/15) Installing libintl (0.19.8.1-r4)
(9/15) Installing libunistring (0.9.10-r0)
(10/15) Installing libxml2 (2.9.9-r2)
(11/15) Installing gettext (0.19.8.1-r4)
(12/15) Installing lua5.3-libs (5.3.5-r2)
(13/15) Installing libpcap (1.9.0-r1)
(14/15) Installing nmap-ncat (7.70-r4)
(15/15) Installing openssl (1.1.1d-r2)
Executing busybox-1.29.3-r10.trigger
OK: 118 MiB in 69 packages
---> 02b6d8cd12e2
Removing intermediate container b4b349725631
Step 3/15 : ARG version
---> Running in adaf454a9d50
---> 105e11f8a06d
Removing intermediate container adaf454a9d50
Step 4/15 : ENV ROCKETMQ_VERSION ${version}
---> Running in 530b608589d3
---> cc2050818bf1
Removing intermediate container 530b608589d3
Step 5/15 : ENV ROCKETMQ_HOME /home/rocketmq/rocketmq-${ROCKETMQ_VERSION}
---> Running in c33d96bf6c65
---> 99f05c59460c
Removing intermediate container c33d96bf6c65
Step 6/15 : WORKDIR ${ROCKETMQ_HOME}
---> af5a408ef20c
Removing intermediate container d84ddc0708f8
Step 7/15 : RUN set -eux; apk add --virtual .build-deps curl gnupg unzip; curl https://archive.apache.org/dist/rocketmq/${ROCKETMQ_VERSION}/rocketmq-all-${ROCKETMQ_VERSION}-bin-release.zip -o rocketmq.zip; curl https://archive.apache.org/dist/rocketmq/${ROCKETMQ_VERSION}/rocketmq-all-${ROCKETMQ_VERSION}-bin-release.zip.asc -o rocketmq.zip.asc; curl https://www.apache.org/dist/rocketmq/KEYS -o KEYS; gpg --import KEYS; gpg --batch --verify rocketmq.zip.asc rocketmq.zip; unzip rocketmq.zip; mv rocketmq-all*/* . ; rmdir rocketmq-all* ; rm rocketmq.zip rocketmq.zip.asc KEYS; apk del .build-deps ; rm -rf /var/cache/apk/* ; rm -rf /tmp/*
---> Running in cccf01cb11a7
请问现在这个项目现在还有人维护吗?I hava a problem need help。
mac docker做的rocketmq集群(主从),如果我想让docker外部的程序访问 rocketmq,就要配置brokerIP1=docker外部的主机ip。
这虽然是可以让docker外部程序访问rocketmq,但集群内的主从会同步不了,因为slave服务是用的brokerIP1配置的ip访问主服务的。
当然这个也许在linux环境下是可以正常运行的,在mac下需要怎么解决这个问题呢?
The issue tracker is ONLY used for bug report(feature request need to follow RIP process). Keep in mind, please check whether there is an existing same report before your raise a new one.
Alternately (especially if your communication is not a bug report), you can send mail to our mailing lists. We welcome any friendly suggestions, bug fixes, collaboration and other improvements.
Please ensure that your bug report is clear and that it is complete. Otherwise, we may be unable to understand it or to reproduce it, either of which would prevent us from fixing the bug. We strongly recommend the report(bug report or feature request) could include some hints as the following:
BUG REPORT
What did you do (The steps to reproduce)?
What did you expect to see?
What did you see instead?
Please tell us about your environment:
Other information (e.g. detailed explanation, logs, related issues, suggestions how to fix, etc):
FEATURE REQUEST
Please describe the feature you are requesting.
Provide any additional detail on your proposed use case for this feature.
Indicate the importance of this issue to you (blocker, must-have, should-have, nice-to-have). Are you currently using any workarounds to address this issue?
If there are some sub-tasks using -[] for each subtask and create a corresponding issue to map to the sub task:
As ASF required, the old release should be moved to archive service and removed from mirrors.
we can still link to older releases if necessary, but the links should point to the archive server
version: '2'
services:
#Service for nameserver
namesrv:
image: apacherocketmq/rocketmq:4.6.0
container_name: rmqnamesrv
ports:
- 7931:9876
volumes:
- ./data/namesrv/logs:/home/rocketmq/logs
command: sh mqnamesrv
#Service for broker
broker:
image: apacherocketmq/rocketmq:4.6.0
container_name: rmqbroker
links:
- namesrv
ports:
- 7932:10909
- 7933:10911
- 7934:10912
environment:
- NAMESRV_ADDR=namesrv:9876
volumes:
- ./data/broker/logs:/home/rocketmq/logs
- ./data/broker/store:/home/rocketmq/store
- ./data/broker/conf/broker.conf:/opt/rocketmq-4.6.0/conf/broker.conf
command: sh mqbroker -c /opt/rocketmq-4.6.0/conf/broker.conf
rmqconsole:
image: styletang/rocketmq-console-ng
restart: always
container_name: rmqconsole
ports:
- 7935:8080
environment:
JAVA_OPTS: "-Drocketmq.namesrv.addr=namesrv:9876 -Dcom.rocketmq.sendMessageWithVIPChannel=false"
depends_on:
- namesrv
2020-02-03 15:22:47,047 INFO RocketmqRemoting - name server address updated. NEW : [localhost:9876] , OLD: null
2020-02-03 15:22:47,047 INFO RocketmqClient - user specified name server address: localhost:9876
2020-02-03 15:22:47,047 INFO RocketmqClient - Created a new client Instance, InstanceIndex:0, ClientID:192.168.48.2@1580743367427, ClientConfig:ClientConfig [namesrvAddr=localhost:9876, clientIP=192.168.48.2, instanceName=1580743367427, clientCallbackExecutorThreads=4, pollNameServerInterval=30000, heartbeatBrokerInterval=30000, persistConsumerOffsetInterval=5000, pullTimeDelayMillsWhenException=1000, unitMode=false, unitName=null, vipChannelEnabled=false, useTLS=false, language=JAVA, namespace=null], ClientVersion:V4_6_0, SerializerType:JSON
2020-02-03 15:22:47,047 INFO RocketmqClient - Created new MQClientInstance for clientId:[192.168.48.2@1580743367427]
2020-02-03 15:22:47,047 INFO RocketmqCommon - Try to start service thread:PullMessageService started:false lastThread:null
2020-02-03 15:22:47,047 INFO RocketmqCommon - Try to start service thread:RebalanceService started:false lastThread:null
2020-02-03 15:22:47,047 INFO RocketmqClient - PullMessageService service started
2020-02-03 15:22:47,047 INFO RocketmqClient - RebalanceService service started
2020-02-03 15:22:47,047 INFO RocketmqClient - the producer [CLIENT_INNER_PRODUCER] start OK. sendMessageWithVIPChannel=false
2020-02-03 15:22:47,047 INFO RocketmqClient - the client factory [192.168.48.2@1580743367427] start OK
2020-02-03 15:22:47,047 INFO RocketmqClient - the adminExt [admin_ext_group] start OK
2020-02-03 15:22:47,047 INFO RocketmqRemoting - new name server is chosen. OLD: null , NEW: localhost:9876. namesrvIndex = 158
2020-02-03 15:22:47,047 DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
2020-02-03 15:22:47,047 DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
2020-02-03 15:22:47,047 DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.allocator.type: unpooled
2020-02-03 15:22:47,047 DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
2020-02-03 15:22:47,047 DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
2020-02-03 15:22:47,047 DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.threadLocalDirectBufferSize: 65536
2020-02-03 15:22:47,047 DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2020-02-03 15:22:47,047 DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2020-02-03 15:22:47,047 DEBUG io.netty.buffer.ByteBufUtil - -Dio.netty.maxThreadLocalCharBufferSize: 16384
2020-02-03 15:22:47,047 INFO RocketmqRemoting - createChannel: begin to connect remote host[localhost:9876] asynchronously
2020-02-03 15:22:47,047 DEBUG io.netty.util.internal.JavassistTypeParameterMatcherGenerator - Generated: io.netty.util.internal.matchers.org.apache.rocketmq.remoting.protocol.RemotingCommandMatcher
2020-02-03 15:22:47,047 DEBUG io.netty.util.internal.JavassistTypeParameterMatcherGenerator - Generated: io.netty.util.internal.matchers.org.apache.rocketmq.remoting.protocol.RemotingCommandMatcher
2020-02-03 15:22:47,047 DEBUG io.netty.util.internal.JavassistTypeParameterMatcherGenerator - Generated: io.netty.util.internal.matchers.org.apache.rocketmq.remoting.protocol.RemotingCommandMatcher
2020-02-03 15:22:47,047 INFO RocketmqRemoting - NETTY CLIENT PIPELINE: CONNECT UNKNOWN => ocalhost/127.0.0.1:9876
2020-02-03 15:22:47,047 INFO RocketmqRemoting - createChannel: connect remote host[localhost:9876] success, DefaultChannelPromise@7e07db1f(success)
2020-02-03 15:22:47,047 ERROR RocketmqAuthorize - [10015:signature-failed] unable to calculate a request signature. error=Algorithm HmacSHA1 not available
java.security.NoSuchAlgorithmException: Algorithm HmacSHA1 not available
at javax.crypto.Mac.getInstance(Mac.java:181)
at org.apache.rocketmq.acl.common.AclSigner.sign(AclSigner.java:57)
at org.apache.rocketmq.acl.common.AclSigner.signAndBase64Encode(AclSigner.java:79)
at org.apache.rocketmq.acl.common.AclSigner.calSignature(AclSigner.java:73)
at org.apache.rocketmq.acl.common.AclSigner.calSignature(AclSigner.java:68)
at org.apache.rocketmq.acl.common.AclUtils.calSignature(AclUtils.java:69)
at org.apache.rocketmq.acl.common.AclClientRPCHook.doBeforeRequest(AclClientRPCHook.java:44)
at org.apache.rocketmq.remoting.netty.NettyRemotingAbstract.doBeforeRpcHooks(NettyRemotingAbstract.java:172)
at org.apache.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(NettyRemotingClient.java:368)
at org.apache.rocketmq.client.impl.MQClientAPIImpl.getBrokerClusterInfo(MQClientAPIImpl.java:1337)
at org.apache.rocketmq.tools.admin.DefaultMQAdminExtImpl.examineBrokerClusterInfo(DefaultMQAdminExtImpl.java:306)
at org.apache.rocketmq.tools.admin.DefaultMQAdminExt.examineBrokerClusterInfo(DefaultMQAdminExt.java:251)
at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.printClusterBaseInfo(ClusterListSubCommand.java:172)
at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.execute(ClusterListSubCommand.java:88)
at org.apache.rocketmq.tools.command.MQAdminStartup.main0(MQAdminStartup.java:139)
at org.apache.rocketmq.tools.command.MQAdminStartup.main(MQAdminStartup.java:90)
I start a rocketmq continar with docker-compose.yml for test , and it took 8G memory by default. How should i do to limited memory use to 512M ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.