jbvmio / kafkactl Goto Github PK
View Code? Open in Web Editor NEWCLI for Apache Kafka Management -
Home Page: http://www.jbvm.io/kafkactl
License: Apache License 2.0
CLI for Apache Kafka Management -
Home Page: http://www.jbvm.io/kafkactl
License: Apache License 2.0
when attempting to build today on go 1.13, I get the following dependency error:
go build -o $GOPATH/bin/kafkactl
build github.com/jbvmio/kafkactl: cannot load github.com/gizak/termui: module github.com/gizak/termui@latest (v3.1.0+incompatible) found, but does not contain package github.com/gizak/termui
I followed the instructions provided on the README file of this repository. I installed golang 1.19, from Kali's APT repository. Not quite sure what the next steps are here.
PWD: /home/user/projects/tmp/kafkactl
GOPATH: /home/user/go
❯ go build -o $GOPATH/bin/kafkactl
cli/kafka/adminConfig.go:24:2: github.com/jbvmio/[email protected]: replacement directory ../kafka does not exist
cli/cmd/root.go:28:2: github.com/jbvmio/[email protected] (replaced by ../kafka): reading ../kafka/go.mod: open /home/user/projects/tmp/kafka/go.mod: no such file or directory
It would be great to have additional flag for the kafkactl consume
to display pretty printed JSON.
Raw JSON:
{"type":"UserStatusChanged","id":"e886eb90-4003-4b1f-9971-8f4bdb23dc89","timestamp":1588167444,"data":{"user":{"id":331,"status":"approved"},"from_status":"created","to_status":"approved","changed_at":"2020-04-29T13:37:18+00:00"}}
Pretty printed JSON:
{
"type": "UserStatusChanged",
"id": "e886eb90-4003-4b1f-9971-8f4bdb23dc89",
"timestamp": 1588167444,
"data": {
"user": {
"id": 331,
"status": "approved"
},
"from_status": "created",
"to_status": "approved",
"changed_at": "2020-04-29T13:37:18+00:00"
}
}
It might be useful on development phase for debugging the messages.
When I try to install under homebrew, I get the following error:
❱ brew tap jbvmio/tap
==> Tapping jbvmio/tap
Cloning into '/opt/homebrew/Library/Taps/jbvmio/homebrew-tap'...
remote: Enumerating objects: 41, done.
remote: Counting objects: 100% (41/41), done.
remote: Compressing objects: 100% (30/30), done.
remote: Total 179 (delta 19), reused 27 (delta 11), pack-reused 138
Receiving objects: 100% (179/179), 1.12 MiB | 8.75 MiB/s, done.
Resolving deltas: 100% (67/67), done.
Error: Invalid formula: /opt/homebrew/Library/Taps/jbvmio/homebrew-tap/Formula/kafkactl.rb
kafkactl: /opt/homebrew/Library/Taps/jbvmio/homebrew-tap/Formula/kafkactl.rb:28: syntax error, unexpected end-of-input, expecting end
Error: Cannot tap jbvmio/tap: invalid syntax in tap!
Hi,
I can't figure out how --json-filter
is supposed to work to filter the messages as I follow a topic.
Could you, please, show an example?
Thank you!
A lot of kafka users will probably be on the Confluent Platform and dealing with Avro and the Schema Registry.
Using curl to interact with the Schema Registry is cumbersome and doesn't produce the most readable output.
Schema Registry support would be a neat addition to this tool.
Thanks for creating this useful utility!
One feature that seems missing is creating topics with An initial configuration
example usage could possibly look like
kafkactl --broker brokerhost01:9092 \
admin create topic newtopic01 \
--partitions 15 \
--rfactor 3 \
--config cleanup.policy=compact \
--config min.compaction.lag.ms=0 \
--config delete.retention.ms=604800000 \
--config min.cleanable.dirty.ratio=0.05
Blocked by jbvmio/kafka#1
brew install jbvmio/tap/kafkactl
Error: jbvmio/tap/kafkactl: wrong number of arguments (given 1, expected 0)
Hello.
I am newbie and try to use kafkactl:
podman run -it --rm --name kafkactl -v ~/.config/kafkactl/config.yml:/etc/kafkactl/config.yml \
docker.io/deviceinsight/kafkactl:v1.11.1 \
kafkactl --verbose get topics
[kafkactl] 2020/09/22 09:02:18 Using config file: /etc/kafkactl/config.yml
[kafkactl] 2020/09/22 09:02:18 Assuming kafkaVersion: 2.0.0
[kafkactl] 2020/09/22 09:02:18 Assuming kafkaVersion: 2.0.0
[sarama ] 2020/09/22 09:02:18 Initializing new client
[sarama ] 2020/09/22 09:02:18 client/metadata fetching metadata for all topics from broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:02:18 Connected to broker at ecsc00a060af.epam.com:9092 (unregistered)
[sarama ] 2020/09/22 09:02:48 client/metadata got error from broker -1 while fetching metadata: read tcp 10.0.2.100:44214->10.6.194.60:9092: i/o timeout
[sarama ] 2020/09/22 09:02:48 Closed connection to broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:02:48 client/metadata no available broker to send metadata request to
[sarama ] 2020/09/22 09:02:48 client/brokers resurrecting 1 dead seed brokers
[sarama ] 2020/09/22 09:02:48 client/metadata retrying after 250ms... (3 attempts remaining)
[sarama ] 2020/09/22 09:02:48 client/metadata fetching metadata for all topics from broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:02:48 Connected to broker at ecsc00a060af.epam.com:9092 (unregistered)
[sarama ] 2020/09/22 09:03:18 client/metadata got error from broker -1 while fetching metadata: read tcp 10.0.2.100:44222->10.6.194.60:9092: i/o timeout
[sarama ] 2020/09/22 09:03:18 Closed connection to broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:03:18 client/metadata no available broker to send metadata request to
[sarama ] 2020/09/22 09:03:18 client/brokers resurrecting 1 dead seed brokers
[sarama ] 2020/09/22 09:03:18 client/metadata retrying after 250ms... (2 attempts remaining)
[sarama ] 2020/09/22 09:03:18 client/metadata fetching metadata for all topics from broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:03:18 Connected to broker at ecsc00a060af.epam.com:9092 (unregistered)
[sarama ] 2020/09/22 09:03:48 client/metadata got error from broker -1 while fetching metadata: read tcp 10.0.2.100:44228->10.6.194.60:9092: i/o timeout
[sarama ] 2020/09/22 09:03:48 Closed connection to broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:03:48 client/metadata no available broker to send metadata request to
[sarama ] 2020/09/22 09:03:48 client/brokers resurrecting 1 dead seed brokers
[sarama ] 2020/09/22 09:03:48 client/metadata retrying after 250ms... (1 attempts remaining)
[sarama ] 2020/09/22 09:03:49 client/metadata fetching metadata for all topics from broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:03:49 Connected to broker at ecsc00a060af.epam.com:9092 (unregistered)
[sarama ] 2020/09/22 09:04:19 client/metadata got error from broker -1 while fetching metadata: read tcp 10.0.2.100:44244->10.6.194.60:9092: i/o timeout
[sarama ] 2020/09/22 09:04:19 Closed connection to broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:04:19 client/metadata no available broker to send metadata request to
[sarama ] 2020/09/22 09:04:19 client/brokers resurrecting 1 dead seed brokers
[sarama ] 2020/09/22 09:04:19 Closing Client
failed to create cluster admin: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
Config is very simple (sed to show just non-comments and non-empty lines):
$ sed 's/#.*$//;/^$/d' .kafkactl.yaml
contexts:
sandbox:
name: sandbox
brokers:
- ecsc00a060af.epam.com:9092
current-context: sandbox
And according to wireshark
dump TCP session kafkactl
send metadata request and can connect to the server, but newer receive response:
Where I am wrong?
I'm trying to reset the connect consumer group offset and get an error.
Command: kafkactl reset cgo --oldest -t users connect-cluster
Error: kafka server: The provider group protocol type is incompatible with the other members.
I've been using kafkactl setting up a kafka cluster and finding it very useful, thanks a bunch.
Now I want to put this thing in production, and that includes only listening on SSL.
But unless I'm missing something, it seems kafkactl will only connect to brokers over plaintext.
Is SSL support something that's on the roadmap? The todo lists Add Security Features but that could be many things.
Thank you for considering adding SSL support.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.