bbaugher / confluent Goto Github PK

@bbaugher we use confluent cookbook in our repos as core and make necessary changes in local.
We provide "depends 'confluent'" in our metadata.rb but recent changes are not getting reflecting after chef build and run. Can you please suggest on how we can pull recent changes.

Starting and stopping the service does not work. The class attribute is wrong and should be changed from

default["confluent"]["schema-registry"]["class"] = "io.confluent.kafka.schemaregistry.rest.Main"

to

default["confluent"]["schema-registry"]["class"] = "io.confluent.kafka.schemaregistry.rest.SchemaRegistryMain"

which seems to work.

Sorry due to company policy I cannot commit to open source projects hence the issue.

Currently we install the confluent package into /opt/confluent/confluent-VERSION. As more versions are installed this takes up more space but we likely don't need to maintain all our other versions

In the /etc/init.d/kafka script, the findPid() function uses java class to identify the kafka process:

CLASS=kafka.Kafka

findPid() {
  FOUND_PID=`pgrep -u $USER -f $CLASS`
}

However, the java class listed in the command line for the kafka process is:

io.confluent.support.metrics.SupportedKafka

This prevents service commands for kafka from functioning properly. Modifying the class name in the init.d script resolves this issue.

Issue identified on confluent version 4.1.1. Have not tested on other versions.

In an early version of confluent the start script had a bug where it wouldn't start so we packaged the fix in our cookbook. It was only supposed to be applied to 1.0 but was actually applied to everything else

As we like and use this cookbook in production, I was curious if there was a way that you could add topic creations support within this cookbook. It would help support the managing the cluster from cookbook perspective.

Things that would be helpful are these things

• topic name
• replication factor
• other topic configuration options

It's a bad practices to derived attributes in default.rb because it's not recompute when you run the chef-client.

It will be a better place to put this code in recipes/_install.rb

version_numbers = node["confluent"]["version"].split(".")
archive_version = "#{version_numbers[0]}.#{version_numbers[1]}"

default["confluent"]["artifact_url"] = File.join("http://packages.confluent.io/archive", archive_version, "confluent-#{node["confluent"]["version"]}-#{node["confluent"]["scala_version"]}.zip")

See Avoid derived attributes in attribute files in
https://github.com/pulseenergy/chef-style-guide#attributes

As jars are deployed to connect via node['confluent']['kafka-connect']['jar_urls'] they are written to a confluent shared lib directory. These jars could change (update version or remove) and the JAR would still be in the shared directory. These JARs should be cleaned up so they are removed from connect's classpath

Almost all template resources have backup false.

Could this be updated to be an attribute, defaulting to false, for those that would like to keep a backup of what changed?

Hi,

Both Kafka-rest and Schema-registry do not start after a reboot with Ubuntu 14.04 (first noticed after an image was created). They fail because Kafka at this time has not finished starting. Have tried forcing the priority with little effect.

It would also be useful to disable auto start of these services via attributes as I want to get them to use the auto broker ids which isn't possible due to the configuration.

Is the next release going to be out soon? I'd like to be able to use the recipes for kafka and schema registry. I'm cloning the repository at master but I'd like to be able to pin a version.