Comments (20)
calvn
commented on July 24, 2024
Can you provide procedure (i.e. commands) and the logs?
from git2consul.
juicedM3
commented on July 24, 2024
First run:
node . -e consul.service.consul -p 8500 -t $MASTER_TOKEN --config-file /tmp/git2consul.json
Second run:
node . -e consul.service.consul -p 8500 -t $MASTER_TOKEN
No errors. Doesn't even see the new repo until the 2nd run.
from git2consul.
calvn
commented on July 24, 2024
What is your the new repo that you are referring to, and can you also provide the git2consul.json?
Edit: I see that on the second run, there is the additional "sample_configuration" repo, is that the one you are referring to?
from git2consul.
calvn
commented on July 24, 2024
You probably have to include "halt_on_change": true in your configuration, and restart the service through some mechanism if the configuration is changed/updated.
Refer to: https://github.com/Cimpress-MCP/git2consul#halt-on-change
from git2consul.
juicedM3
commented on July 24, 2024
I'm pointing at the git2consul sample repo. You could probably use that repo twice in one config file with a different name and see the same results.
{
"version": "1.0",
"local_store": "/tmp/git2consul_cache",
"logger" : {
"name" : "git2consul",
"streams" : [
{
"level": "trace",
"stream": "process.stdout"
},
{
"level": "debug",
"type": "rotating-file",
"path": "/tmp/git2consul.log"
}
]
},
"repos": [{
"name": "logging2",
"url": "ssh://[email protected]/logging.git",
"expand_keys": true,
"branches": ["master"],
"hooks": [{
"type": "stash",
"port": "5050",
"url": "/gitpoke"
},
{
"type": "polling",
"interval": "1"
}]
},
{
"name": "filebeat",
"url": "ssh://[email protected]/logging.git",
"expand_keys": true,
"source_root": "filebeat/etc",
"branches": ["master"],
"hooks": [{
"type": "stash",
"port": "5051",
"url": "/gitpoke"
},
{
"type": "polling",
"interval": "1"
}]
},
{
"name": "fifo2kafka",
"url": "ssh://[email protected]/logging.git",
"expand_keys": true,
"source_root": "fifo2kafka/etc",
"branches": ["master"],
"hooks": [{
"type": "stash",
"port": "5052",
"url": "/gitpoke"
},
{
"type": "polling",
"interval": "1"
}]
},
{
"name": "kafkabeat",
"url": "ssh://[email protected]/logging.git",
"expand_keys": true,
"source_root": "kafkabeat/etc",
"branches": ["master"],
"hooks": [{
"type": "stash",
"port": "5053",
"url": "/gitpoke"
},
{
"type": "polling",
"interval": "1"
}]
},
{
"name" : "sample_configuration",
"url" : "https://github.com/ryanbreen/git2consul_data.git",
"branches" : ["dev"],
"hooks": [{
"type" : "polling",
"interval" : "1"
}]
}]
}
So for clarification, git2consul is already up and running. I control-C to stop it. I make the config file change and then restart it (first run referenced above). I see that it doesn't pick up the new changes and control-C out of it a 2nd time. I restart it and then see the changes (second run referenced above). Maybe halt_on_change would help.
from git2consul.
calvn
commented on July 24, 2024
Logs on first run:
{"name":"git2consul","hostname":"ab97ce30e784","pid":1,"level":30,"msg":"Adding /etc/git2consul.d/config.json to KV git2consul/config as: \n{\n \"version\": \"1.0\",\n \"repos\" : [{\n \"name\" : \"sample_configuration\",\n \"url\" : \"https://github.com/ryanbreen/git2consul_data.git\",\n \"branches\" : [\"dev\"],\n \"hooks\": [{\n \"type\" : \"polling\",\n \"interval\" : \"1\"\n }]\n }]\n}\n","time":"2016-03-28T21:07:41.049Z","v":0}
{"name":"git2consul","hostname":"ab97ce30e784","pid":1,"level":30,"msg":"git2consul is running","time":"2016-03-28T21:07:41.266Z","v":0}
{"name":"git2consul","hostname":"ab97ce30e784","pid":1,"level":30,"msg":"Initting repo sample_configuration","time":"2016-03-28T21:07:41.273Z","v":0}
{"name":"git2consul","hostname":"ab97ce30e784","pid":1,"level":30,"msg":"Initting branch /tmp/sample_configuration /tmp/sample_configuration/dev","time":"2016-03-28T21:07:41.273Z","v":0}
{"name":"git2consul","hostname":"ab97ce30e784","pid":1,"level":40,"msg":"Purging branch cache /tmp/sample_configuration/dev for branch dev in repo sample_configuration","time":"2016-03-28T21:07:41.274Z","v":0}
{"name":"git2consul","hostname":"ab97ce30e784","pid":1,"level":30,"msg":"Initialized branch dev from sample_configuration","time":"2016-03-28T21:07:41.613Z","v":0}
{"name":"git2consul","hostname":"ab97ce30e784","pid":1,"level":30,"msg":"Loaded repo sample_configuration","time":"2016-03-28T21:07:41.615Z","v":0}Logs on second run:
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"Adding /etc/git2consul.d/config.json to KV git2consul/config as: \n{\n \"version\": \"1.0\",\n \"repos\" : [{\n \"name\" : \"sample_configuration\",\n \"url\" : \"https://github.com/ryanbreen/git2consul_data.git\",\n \"branches\" : [\"dev\"],\n \"hooks\": [{\n \"type\" : \"polling\",\n \"interval\" : \"1\"\n }]\n },{\n \"name\" : \"sample_configuration2\",\n \"url\" : \"https://github.com/ryanbreen/git2consul_data.git\",\n \"branches\" : [\"dev\"],\n \"hooks\": [{\n \"type\" : \"polling\",\n \"interval\" : \"1\"\n }]\n }]\n}\n","time":"2016-03-28T21:09:53.559Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"git2consul is running","time":"2016-03-28T21:09:53.777Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"Initting repo sample_configuration","time":"2016-03-28T21:09:53.782Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"Initting branch /tmp/sample_configuration /tmp/sample_configuration/dev","time":"2016-03-28T21:09:53.782Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"Initting repo sample_configuration2","time":"2016-03-28T21:09:53.783Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"Initting branch /tmp/sample_configuration2 /tmp/sample_configuration2/dev","time":"2016-03-28T21:09:53.783Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":40,"msg":"Purging branch cache /tmp/sample_configuration/dev for branch dev in repo sample_configuration","time":"2016-03-28T21:09:53.784Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":40,"msg":"Purging branch cache /tmp/sample_configuration2/dev for branch dev in repo sample_configuration2","time":"2016-03-28T21:09:53.785Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"Initialized branch dev from sample_configuration2","time":"2016-03-28T21:09:54.139Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"Loaded repo sample_configuration2","time":"2016-03-28T21:09:54.140Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"Initialized branch dev from sample_configuration","time":"2016-03-28T21:09:54.262Z","v":0}
{"name":"git2consul","hostname":"f12864c69f74","pid":1,"level":30,"msg":"Loaded repo sample_configuration","time":"2016-03-28T21:09:54.263Z","v":0}KV on Consul after second run:
curl "127.0.0.1:8500/v1/kv/?recurse&pretty"
[
{
"CreateIndex": 14,
"ModifyIndex": 26,
"LockIndex": 0,
"Key": "git2consul/config",
"Flags": 0,
"Value": "ewogICJ2ZXJzaW9uIjogIjEuMCIsCiAgInJlcG9zIiA6IFt7CiAgICAibmFtZSIgOiAic2FtcGxlX2NvbmZpZ3VyYXRpb24iLAogICAgInVybCIgOiAiaHR0cHM6Ly9naXRodWIuY29tL3J5YW5icmVlbi9naXQyY29uc3VsX2RhdGEuZ2l0IiwKICAgICJicmFuY2hlcyIgOiBbImRldiJdLAogICAgImhvb2tzIjogW3sKICAgICAgInR5cGUiIDogInBvbGxpbmciLAogICAgICAiaW50ZXJ2YWwiIDogIjEiCiAgICB9XQogIH0sewogICAgIm5hbWUiIDogInNhbXBsZV9jb25maWd1cmF0aW9uMiIsCiAgICAidXJsIiA6ICJodHRwczovL2dpdGh1Yi5jb20vcnlhbmJyZWVuL2dpdDJjb25zdWxfZGF0YS5naXQiLAogICAgImJyYW5jaGVzIiA6IFsiZGV2Il0sCiAgICAiaG9va3MiOiBbewogICAgICAidHlwZSIgOiAicG9sbGluZyIsCiAgICAgICJpbnRlcnZhbCIgOiAiMSIKICAgIH1dCiAgfV0KfQo="
},
{
"CreateIndex": 18,
"ModifyIndex": 36,
"LockIndex": 0,
"Key": "sample_configuration/dev.ref",
"Flags": 0,
"Value": "NTI0YTAyMWUzYjQ0YzE0N2FmNjVjNzM5Y2FhMGQ3ZjY0NjNhNDkyOA=="
},
{
"CreateIndex": 19,
"ModifyIndex": 33,
"LockIndex": 0,
"Key": "sample_configuration/dev/sample.conf",
"Flags": 0,
"Value": "c2FtcGxlID0gY29uZmlndXJhdGlvbg=="
},
{
"CreateIndex": 20,
"ModifyIndex": 34,
"LockIndex": 0,
"Key": "sample_configuration/dev/sample.json",
"Flags": 0,
"Value": "ewoJInNhbXBsZSI6ICJjb25maWd1cmF0aW9uIgp9"
},
{
"CreateIndex": 21,
"ModifyIndex": 35,
"LockIndex": 0,
"Key": "sample_configuration/dev/sample.yaml",
"Flags": 0,
"Value": "c2FtcGxlOiBjb25maWd1cmF0aW9u"
},
{
"CreateIndex": 28,
"ModifyIndex": 32,
"LockIndex": 0,
"Key": "sample_configuration2/dev.ref",
"Flags": 0,
"Value": "NTI0YTAyMWUzYjQ0YzE0N2FmNjVjNzM5Y2FhMGQ3ZjY0NjNhNDkyOA=="
},
{
"CreateIndex": 29,
"ModifyIndex": 29,
"LockIndex": 0,
"Key": "sample_configuration2/dev/sample.conf",
"Flags": 0,
"Value": "c2FtcGxlID0gY29uZmlndXJhdGlvbg=="
},
{
"CreateIndex": 31,
"ModifyIndex": 31,
"LockIndex": 0,
"Key": "sample_configuration2/dev/sample.json",
"Flags": 0,
"Value": "ewoJInNhbXBsZSI6ICJjb25maWd1cmF0aW9uIgp9"
},
{
"CreateIndex": 30,
"ModifyIndex": 30,
"LockIndex": 0,
"Key": "sample_configuration2/dev/sample.yaml",
"Flags": 0,
"Value": "c2FtcGxlOiBjb25maWd1cmF0aW9u"
}
]Edit: This is from a clean run, the first run is the actual first time that git2consul is running and populates git2consul/config. The second run (which to my understand would be the first run in your case) is with the updated configuration, which successfully updates to git2consul/config and pulls all the repos on the same run without the need for a restart.
from git2consul.
juicedM3
commented on July 24, 2024
Started fresh on a different server. Cleared out all git2consul data in Consul. Configuration:
{
"version": "1.0",
"local_store": "/tmp/git2consul_cache",
"halt_on_change": true,
"logger" : {
"name" : "git2consul",
"streams" : [
{
"level": "trace",
"stream": "process.stdout"
},
{
"level": "debug",
"type": "rotating-file",
"path": "/tmp/git2consul.log"
}
]
},
"repos": [{
"name" : "sample_configuration",
"url" : "https://github.com/ryanbreen/git2consul_data.git",
"branches" : ["dev"],
"hooks": [{
"type" : "polling",
"interval" : "1"
}]
}]
}
First run:
$ node . -e consul.service.consul -p 8500 -t $CONSUL_TOKEN --config-file /tmp/git2consul.json
{"name":"git2consul","hostname":"host.example.com","pid":21525,"level":30,"msg":"Adding /tmp/git2consul.json to KV git2consul/config as: \n{\n \"version\": \"1.0\",\n \"local_store\": \"/tmp/git2consul_cache\",\n \"halt_on_change\": true,\n \"logger\" : {\n \"name\" : \"git2consul\",\n \"streams\" : [\n {\n \"level\": \"trace\",\n \"stream\": \"process.stdout\"\n },\n {\n \"level\": \"trace\",\n \"type\": \"rotating-file\",\n \"path\": \"/tmp/git2consul.log\"\n }\n ]\n },\n \"repos\": [{\n \"name\" : \"sample_configuration\",\n \"url\" : \"https://github.com/ryanbreen/git2consul_data.git\",\n \"branches\" : [\"dev\"],\n \"hooks\": [{\n \"type\" : \"polling\",\n \"interval\" : \"1\"\n }]\n }]\n}\n","time":"2016-03-29T13:57:45.956Z","v":0}
Config value is not valid JSON: undefined
Second run:
$ node . -e consul.service.consul -p 8500 -t $CONSUL_TOKEN
git2consul starts and sample_configuration is loaded. second_run.txt.
Control-C to shutdown. Update config file to add a 2nd repo:
{
"version": "1.0",
"local_store": "/tmp/git2consul_cache",
"halt_on_change": true,
"logger" : {
"name" : "git2consul",
"streams" : [
{
"level": "trace",
"stream": "process.stdout"
},
{
"level": "trace",
"type": "rotating-file",
"path": "/tmp/git2consul.log"
}
]
},
"repos": [{
"name" : "sample_configuration",
"url" : "https://github.com/ryanbreen/git2consul_data.git",
"branches" : ["dev"],
"hooks": [{
"type" : "polling",
"interval" : "1"
}]
},
{
"name" : "sample_configuration2",
"url" : "https://github.com/ryanbreen/git2consul_data.git",
"branches" : ["dev"],
"hooks": [{
"type" : "polling",
"interval" : "1"
}]
}]
}
Next run:
$ node . -e consul.service.consul -p 8500 -t $CONSUL_TOKEN --config-file /tmp/git2consul.json
sample_configuration2 is not being loaded. third_run.txt.
Control-C to shutdown. Last run, no changes:
$ node . -e consul.service.consul -p 8500 -t $CONSUL_TOKEN
sample_configuration2 is now loaded and everything is good. forth_run.txt
from git2consul.
calvn
commented on July 24, 2024
This looks weird right off the bat. On your first run git2consul says the following: Config value is not valid JSON: undefined. However, on your second run the config loads even though you didn't provide a config file.
Can you verify that you are running v0.12.9?
from git2consul.
juicedM3
commented on July 24, 2024
On the first run, the config file is loaded into Consul but then git2consul dies. I am running v0.12.9. In the package.json is see [email protected] plus I see your changes in lib/config_reader.js. Is it another ACL issue? I'll disable ACL in Consul and try again.
from git2consul.
juicedM3
commented on July 24, 2024
It's an ACL issue. I gave the anonymous token full read/write access to all keys and git2consul behaves as expected.
from git2consul.
calvn
commented on July 24, 2024
Thanks for the triage. Can you try passing the token as an environment variable? I am wondering if we missed something during the last patch.
from git2consul.
juicedM3
commented on July 24, 2024
Passing the token as an environment variable does not make any difference. Same error on initial start.
from git2consul.
calvn
commented on July 24, 2024
I am able to repro the case were I get Config value is not valid JSON: undefined from a fresh Consul server. Interestingly though the example provided in Cimpress-MCP/docker-git2consul does not trigger such error.
from git2consul.
calvn
commented on July 24, 2024
consul.kv.get() returns an empty result on the first run. I suspect this is the underlying cause of this issue.
https://github.com/Cimpress-MCP/git2consul/blob/master/lib/config_reader.js#L20
from git2consul.
calvn
commented on July 24, 2024
Actually I am getting Config value is not valid JSON: undefined on the container as well.
from git2consul.
calvn
commented on July 24, 2024
It looks like config_reader.read() is called way too fast, before consul has the chance to populate the KV. I tested by having config_reader.read() inside of setTimeout()'s callback and it works on first run.
from git2consul.
calvn
commented on July 24, 2024
@juicedM3 version 0.12.10 should provide a fix for this.
from git2consul.
juicedM3
commented on July 24, 2024
@cleung2010: Just tested this. The initial starting issue is fixed along with updating the configuration and restarting. Thanks! It may have fixed another issue I was seeing. Nothing was happening when I updated the configuration in Consul even with halt_on_change set to true. Now when I change the config in Consul, git2consul does halt.
from git2consul.
calvn
commented on July 24, 2024
Cool, feel free to close the issue if it's resolved.
from git2consul.
juicedM3
commented on July 24, 2024
Fixed in #70. Closing.
from git2consul.
Related Issues (20)
- Git2consul Generates PUT GET every 2 seconds on local agent. Why is that?
- How to configure parameters to use consul with HTTPS? HOT 1
- Problem synchronizing binary files
- git2consul is not converting .json file to KV entry in consul HOT 3
- git2consul problem does not cause Docker container to exit HOT 4
- Project status, new official fork? HOT 6
- Getting [Error: consul: kv.get: connect ECONNREFUSED] HOT 1
- Update of git files trigger unwanted delete of similar key name in consul
- support unix socket?
- files in git not within source_root forcing check_pending to setLastProcessedRef HOT 1
- Files not in source_root are killing my consul API. HOT 1
- support_tags should have option for "true", "false", and "read_once"
- git2consul not importing list in yaml HOT 1
- Git2Consul tool not supported for Google Cloud Run
- Windows - running example sync content of git2consul.json intro Consul KV
- readme - link to gitlab docs is broken
- a key for an annotation with slash in the name
- Is there a way to run git2consul as a CLI tool and not as a service? HOT 2
- panic: runtime error: invalid memory address or nil pointer dereference
- reference not found error
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from git2consul.