dangdangdotcom / config-toolkit Goto Github PK

View Code? Open in Web Editor NEW

497.0 497.0 314.0 499 KB

Config toolkit用于简化从本地配置文件到zookeeper的迁移

License: Apache License 2.0

Java 81.42% CSS 1.75% JavaScript 5.84% FreeMarker 10.98%

config-toolkit's People

Contributors

Stargazers

Watchers

Forkers

doctorwho1986 joliny zipu888 qiushuizy feiyu2016 ddy88958620 peidachang jjshanwei liuyindong majinkai buddygjw shanyechen kangfoo humyna lewis-ing xieyuan1988 yab gitriver sutengfang hujihong sidneydev chenld hanwangkun wellpeny hope-bravery zcrocke adonis2014 yonglehou ryan0923 stvliu wklong boy12371 fysoft2006 jimmyzhong debenson zhanghua498 kangzhenkang zhaoxunyong zhzhy917 dennisit lacrazyboy love61v rbible sdgdsffdsfff lzfcome nevermind2014 zhangwei5095 fengshao0907 lsp1357 chakra-coder strivezxq dadarom eisig quyixia xhjftech scwanglijun excellme yangyucs shangyi0102 suqcnn haoqoo czdreams jacklcz moujian wuyingminhui kuguobing jiezzy xiaonanforever amsalt wangjianbao ericchangzmm caowenkun khaliyo chenkh phantomes kris1985 sqshine k4124k kevingzhz hadoop835 luguangl redbeans2015 tsw424 happygao521 mlc0202 joseanluo zeq9069 vniu xinmiao0377 shylockgou lejingw yuyunyao fnet123 zwye2008 rucky2013 lifewei jiangjunlong holysky5 sixgodx jiangangch

config-toolkit's Issues

当运行在有session共享的环境下时会有以下异常

当运行在有session共享的环境下时会有以下异常(tomcat7)

session共享方案
https://github.com/lenxeon/notes/tree/master/%E8%BF%90%E7%BB%B4/201512/tomcat7%E9%9B%86%E7%BE%A4session%E5%85%B1%E4%BA%AB

异常
java.lang.NoClassDefFoundError: Lorg/codehaus/janino/ScriptEvaluator;
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2499)
at java.lang.Class.getDeclaredFields(Class.java:1811)
at com.esotericsoftware.kryo.serialize.FieldSerializer.rebuildCachedFields(FieldSerializer.java:61)
at com.esotericsoftware.kryo.serialize.FieldSerializer.(FieldSerializer.java:48)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.(ReferenceFieldSerializer.java:26)
at de.javakaffee.web.msm.serializer.kryo.ReferenceFieldSerializerFactory.newDefaultSerializer(ReferenceFieldSerializerFactory.java:34)
at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder$1.newDefaultSerializer(KryoTranscoder.java:198)
at com.esotericsoftware.kryo.Kryo.newSerializer(Kryo.java:273)
at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder$1.newSerializer(KryoTranscoder.java:176)
at com.esotericsoftware.kryo.Kryo.handleUnregisteredClass(Kryo.java:329)
at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder$1.handleUnregisteredClass(KryoTranscoder.java:193)
at com.esotericsoftware.kryo.Kryo.getRegisteredClass(Kryo.java:305)
at com.esotericsoftware.kryo.Kryo.writeClass(Kryo.java:374)
at de.javakaffee.kryoserializers.ClassSerializer.write(ClassSerializer.java:51)
at de.javakaffee.kryoserializers.ClassSerializer.write(ClassSerializer.java:30)
at com.esotericsoftware.kryo.serialize.SimpleSerializer.writeObjectData(SimpleSerializer.java:17)
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:489)
at com.esotericsoftware.kryo.serialize.CollectionSerializer.writeObjectData(CollectionSerializer.java:86)
at de.javakaffee.kryoserializers.CopyForIterateCollectionSerializer.writeObjectData(CopyForIterateCollectionSerializer.java:46)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.serialize.FieldSerializer.writeObjectData(FieldSerializer.java:175)
at com.esotericsoftware.kryo.serialize.ReferenceFieldSerializer.writeObjectData(ReferenceFieldSerializer.java:52)
at com.esotericsoftware.kryo.Kryo.writeClassAndObject(Kryo.java:489)
at com.esotericsoftware.kryo.serialize.MapSerializer.writeObjectData(MapSerializer.java:104)
at de.javakaffee.kryoserializers.CopyForIterateMapSerializer.writeObjectData(CopyForIterateMapSerializer.java:56)
at com.esotericsoftware.kryo.Serializer.writeObject(Serializer.java:43)
at com.esotericsoftware.kryo.Kryo.writeObject(Kryo.java:520)
at com.esotericsoftware.kryo.ObjectBuffer.writeObject(ObjectBuffer.java:251)
at de.javakaffee.web.msm.serializer.kryo.KryoTranscoder.serializeAttributes(KryoTranscoder.java:303)
at de.javakaffee.web.msm.TranscoderService.serializeAttributes(TranscoderService.java:155)
at de.javakaffee.web.msm.BackupSessionTask.serializeAttributes(BackupSessionTask.java:176)
at de.javakaffee.web.msm.BackupSessionTask.call(BackupSessionTask.java:110)
at de.javakaffee.web.msm.BackupSessionTask.call(BackupSessionTask.java:51)
at de.javakaffee.web.msm.BackupSessionService$SynchronousExecutorService.submit(BackupSessionService.java:346)
at de.javakaffee.web.msm.BackupSessionService.backupSession(BackupSessionService.java:205)
at de.javakaffee.web.msm.MemcachedSessionService.backupSession(MemcachedSessionService.java:1040)
at de.javakaffee.web.msm.RequestTrackingHostValve.backupSession(RequestTrackingHostValve.java:230)
at de.javakaffee.web.msm.RequestTrackingHostValve.invoke(RequestTrackingHostValve.java:159)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:409)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1044)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:607)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:315)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.codehaus.janino.ScriptEvaluator
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1702)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1547)
... 74 more

spring-boot 1.4.4打包成单个jar文件后，"classpath:application.properties" 无法读取

把spring-boot项目打成单个jar包，运行时，无法正确读取配置文件

sprint-boot版本是1.4.4

从配置文件读取ZookeeperSourceFactory的zk配置

如果想要从properties配置文件中读取zk的配置，有没有好的建议？

config-toolkit-demo中的测试不能正确运行

比如WithoutSpring
需要修改Preconditions.checkState("Welcome here.".equals(propertyGroup1.getProperty("string_property_key")));才能正常运行

当网络断线一定时间后，网络恢复。修改配置中心的值后无法在本地获取到更新

在生产上出现过，当服务节点脱网一段时间，在脱网期间配置中心配置修改。
修改后的配置在网络恢复后也无法推送到服务本地了。。
我们内部版本已经做了修改。注册了一个连接状态监听器，当网络恢复后重新注册监听和reload.....
请问，最新版本或许已经解决了这个问题？

关于版本问题

您好一直在用您们的config toolkit 非常好用，也非常感谢。但是类似于 guava curator 版本有点略低是否可以升级下版本非常感谢

操作系统win7 64位中文，jdk1.8.0_11，在config-zk-web工程下运行mvn clean install -Dmaven.test.skip报错

[ERROR] d:\git\config-toolkit\config-zk-web\src\main\java\com\dangdang\config\se
rvice\web\mb\PropertyGroupManagedBean.java:179: 警告: @return 没有说明
[ERROR] * @return
[ERROR] ^
[ERROR] d:\git\config-toolkit\config-zk-web\src\main\java\com\dangdang\config\se
rvice\web\mb\PropertyGroupManagedBean.java:181: 警告: event没有 @param
[ERROR] public void onMenuSelected(SelectEvent event) {
[ERROR] ^
[ERROR] d:\git\config-toolkit\config-zk-web\src\main\java\com\dangdang\config\se
rvice\web\mb\PropertyGroupManagedBean.java:192: 警告: @param 没有说明
[ERROR] * @param event
[ERROR] ^
[ERROR] d:\git\config-toolkit\config-zk-web\src\main\java\com\dangdang\config\se
rvice\web\mb\PropertyGroupManagedBean.java:273: 警告: @param 没有说明
[ERROR] * @param event
[ERROR] ^
[ERROR] d:\git\config-toolkit\config-zk-web\src\main\java\com\dangdang\config\se
rvice\web\mb\PropertyGroupManagedBean.java:303: 警告: @param 没有说明
[ERROR] * @param event
[ERROR] ^

java -version
java version "1.8.0_11"
Java(TM) SE Runtime Environment (build 1.8.0_11-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.11-b03, mixed mode)

已将换行改成CRLF

note:jdk1.7.0_45没有问题，mvn clean install -Dmaven.test.skip能运行成功。

Notifications to observers to async

v2分支监听NodeChildrenChanged事件，并没有处理子节点删除的情况

删除子节点，触发NodeChildrenChanged事件，之后调用ConfigNode#loadKey，加载最新的配置，问题在于properties没有清空，无法正确处理子节点被删除的情况。

有提供http接口修改配置数据吗？

配置数据可能由其他服务动态修改，所以需要一个修改接口，这个功能现在是没有提供的吧？

session expire，不能监听到改动的节点了

重现步骤：

起一个ZookeeperConfigGroup
随便找个地方断点，过一段时间让session超时，看到如下日志：

2018-06-08 15:57:42,398 main-SendThread(x.x.x.x:2181) INFO org.apache.zookeeper.ClientCnxn:975 :: Opening socket connection to server 10.16.2.118/10.16.2.118:2181. Will not attempt to authenticate using SASL (unknown error)
2018-06-08 15:57:42,402 main-SendThread(x.x.x.x:2181) INFO org.apache.zookeeper.ClientCnxn:852 :: Socket connection established to 10.16.2.118/10.16.2.118:2181, initiating session
2018-06-08 15:57:42,405 main-SendThread(x.x.x.x:2181) INFO org.apache.zookeeper.ClientCnxn:1094 :: Unable to reconnect to ZooKeeper service, session 0x36347e4f35801c1 has expired, closing socket connection
2018-06-08 15:57:42,405 main-EventThread WARN org.apache.curator.ConnectionState:289 :: Session expired event received
2018-06-08 15:57:42,406 main-EventThread INFO org.apache.zookeeper.ZooKeeper:438 :: Initiating client connection, connectString=10.16.2.118:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@57bdceaa
2018-06-08 15:57:42,407 main-EventThread INFO org.apache.curator.framework.state.ConnectionStateManager:228 :: State change: LOST
2018-06-08 15:57:42,408 main-SendThread(10.16.2.118:2181) INFO org.apache.zookeeper.ClientCnxn:975 :: Opening socket connection to server 10.16.2.118/10.16.2.118:2181. Will not attempt to authenticate using SASL (unknown error)
2018-06-08 15:57:42,987 main-EventThread INFO org.apache.zookeeper.ClientCnxn:512 :: EventThread shut down

在界面上随便改动一个值，预期程序可以进入以下断点，NodeChildrenChanged或者NodeDataChanged分支：

ConfigNodeEventListener.java

@Override
public void eventReceived(CuratorFramework client, CuratorEvent event) throws Exception {
	if (LOGGER.isDebugEnabled()) {
		LOGGER.debug(event.toString());
	}

	final WatchedEvent watchedEvent = event.getWatchedEvent();
	if (watchedEvent != null) {
		LOGGER.debug("Watched event: {}" + watchedEvent);

		if (watchedEvent.getState() == KeeperState.SyncConnected) {
			boolean someChange = false;
			switch (watchedEvent.getType()) {
			case NodeChildrenChanged:
				configNode.loadNode();
				someChange = true;
				break;
			case NodeDataChanged:
				configNode.reloadKey(watchedEvent.getPath());
				someChange = true;
				break;

结果没有进入。

可能原因：
session失效后，关联在session上的watcher全部被移除。再次重连后，zk node上没有注册watcher,所以改动的node的值，不会触发到eventReceived方法。所以，在重连后，需要调用方法重新在node上注册watcher.

代码上做以下改动case NodeDataChanged后面加一个条件：
case None:
configNode.loadNode();
someChange = false;
break;

loadNode方法会重新注册监听。

建议添加Key和Value的时候自动去除左右的空格（做trim()操作）

很多时候Key和Value都是复制粘贴上去的，总是有粗心的人把末尾的空格粘贴上去。

并且末尾有空格，在配置的页面看不出来。一旦出现问题，比较难定位。

希望能够更新自动trim()的功能，拯救下粗心的程序员。

请问能通过程序修改配置中心的数据，然后刷新到各个分布式服务吗？

Add nodes for specified deployments

为某些特殊的部署节点添加定制的属性