jbvmio / kafkactl Goto Github PK
View Code? Open in Web Editor NEWCLI for Apache Kafka Management -
Home Page: http://www.jbvm.io/kafkactl
License: Apache License 2.0
kafkactl's People
Contributors
Stargazers
Watchers
Forkers
cliffwheadon dmitryilyin go-stream ateijelo zhanglei percyzhang codingknees mschlech showsmallkafkactl's Issues
build broken on termui dependency
when attempting to build today on go 1.13, I get the following dependency error:
go build -o $GOPATH/bin/kafkactl
build github.com/jbvmio/kafkactl: cannot load github.com/gizak/termui: module github.com/gizak/termui@latest (v3.1.0+incompatible) found, but does not contain package github.com/gizak/termui
Error building kafkactl
I followed the instructions provided on the README file of this repository. I installed golang 1.19, from Kali's APT repository. Not quite sure what the next steps are here.
PWD: /home/user/projects/tmp/kafkactl
GOPATH: /home/user/go
❯ go build -o $GOPATH/bin/kafkactl
cli/kafka/adminConfig.go:24:2: github.com/jbvmio/[email protected]: replacement directory ../kafka does not exist
cli/cmd/root.go:28:2: github.com/jbvmio/[email protected] (replaced by ../kafka): reading ../kafka/go.mod: open /home/user/projects/tmp/kafka/go.mod: no such file or directory
JSON Pretty Print
It would be great to have additional flag for the kafkactl consume to display pretty printed JSON.
Raw JSON:
{"type":"UserStatusChanged","id":"e886eb90-4003-4b1f-9971-8f4bdb23dc89","timestamp":1588167444,"data":{"user":{"id":331,"status":"approved"},"from_status":"created","to_status":"approved","changed_at":"2020-04-29T13:37:18+00:00"}}Pretty printed JSON:
{
"type": "UserStatusChanged",
"id": "e886eb90-4003-4b1f-9971-8f4bdb23dc89",
"timestamp": 1588167444,
"data": {
"user": {
"id": 331,
"status": "approved"
},
"from_status": "created",
"to_status": "approved",
"changed_at": "2020-04-29T13:37:18+00:00"
}
}It might be useful on development phase for debugging the messages.
Tap no longer working under homebrew
When I try to install under homebrew, I get the following error:
❱ brew tap jbvmio/tap
==> Tapping jbvmio/tap
Cloning into '/opt/homebrew/Library/Taps/jbvmio/homebrew-tap'...
remote: Enumerating objects: 41, done.
remote: Counting objects: 100% (41/41), done.
remote: Compressing objects: 100% (30/30), done.
remote: Total 179 (delta 19), reused 27 (delta 11), pack-reused 138
Receiving objects: 100% (179/179), 1.12 MiB | 8.75 MiB/s, done.
Resolving deltas: 100% (67/67), done.
Error: Invalid formula: /opt/homebrew/Library/Taps/jbvmio/homebrew-tap/Formula/kafkactl.rb
kafkactl: /opt/homebrew/Library/Taps/jbvmio/homebrew-tap/Formula/kafkactl.rb:28: syntax error, unexpected end-of-input, expecting end
Error: Cannot tap jbvmio/tap: invalid syntax in tap!
--json-filter example?
Hi,
I can't figure out how --json-filter is supposed to work to filter the messages as I follow a topic.
Could you, please, show an example?
Thank you!
Schema Registry Support
A lot of kafka users will probably be on the Confluent Platform and dealing with Avro and the Schema Registry.
Using curl to interact with the Schema Registry is cumbersome and doesn't produce the most readable output.
Schema Registry support would be a neat addition to this tool.
Support topic configuration when creating topics
Thanks for creating this useful utility!
One feature that seems missing is creating topics with An initial configuration
example usage could possibly look like
kafkactl --broker brokerhost01:9092 \
admin create topic newtopic01 \
--partitions 15 \
--rfactor 3 \
--config cleanup.policy=compact \
--config min.compaction.lag.ms=0 \
--config delete.retention.ms=604800000 \
--config min.cleanable.dirty.ratio=0.05
Blocked by jbvmio/kafka#1
Homebrew tap throws error
brew install jbvmio/tap/kafkactl
Error: jbvmio/tap/kafkactl: wrong number of arguments (given 1, expected 0)
client/metadata got error from broker -1 while fetching metadata
Hello.
I am newbie and try to use kafkactl:
podman run -it --rm --name kafkactl -v ~/.config/kafkactl/config.yml:/etc/kafkactl/config.yml \
docker.io/deviceinsight/kafkactl:v1.11.1 \
kafkactl --verbose get topics
[kafkactl] 2020/09/22 09:02:18 Using config file: /etc/kafkactl/config.yml
[kafkactl] 2020/09/22 09:02:18 Assuming kafkaVersion: 2.0.0
[kafkactl] 2020/09/22 09:02:18 Assuming kafkaVersion: 2.0.0
[sarama ] 2020/09/22 09:02:18 Initializing new client
[sarama ] 2020/09/22 09:02:18 client/metadata fetching metadata for all topics from broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:02:18 Connected to broker at ecsc00a060af.epam.com:9092 (unregistered)
[sarama ] 2020/09/22 09:02:48 client/metadata got error from broker -1 while fetching metadata: read tcp 10.0.2.100:44214->10.6.194.60:9092: i/o timeout
[sarama ] 2020/09/22 09:02:48 Closed connection to broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:02:48 client/metadata no available broker to send metadata request to
[sarama ] 2020/09/22 09:02:48 client/brokers resurrecting 1 dead seed brokers
[sarama ] 2020/09/22 09:02:48 client/metadata retrying after 250ms... (3 attempts remaining)
[sarama ] 2020/09/22 09:02:48 client/metadata fetching metadata for all topics from broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:02:48 Connected to broker at ecsc00a060af.epam.com:9092 (unregistered)
[sarama ] 2020/09/22 09:03:18 client/metadata got error from broker -1 while fetching metadata: read tcp 10.0.2.100:44222->10.6.194.60:9092: i/o timeout
[sarama ] 2020/09/22 09:03:18 Closed connection to broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:03:18 client/metadata no available broker to send metadata request to
[sarama ] 2020/09/22 09:03:18 client/brokers resurrecting 1 dead seed brokers
[sarama ] 2020/09/22 09:03:18 client/metadata retrying after 250ms... (2 attempts remaining)
[sarama ] 2020/09/22 09:03:18 client/metadata fetching metadata for all topics from broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:03:18 Connected to broker at ecsc00a060af.epam.com:9092 (unregistered)
[sarama ] 2020/09/22 09:03:48 client/metadata got error from broker -1 while fetching metadata: read tcp 10.0.2.100:44228->10.6.194.60:9092: i/o timeout
[sarama ] 2020/09/22 09:03:48 Closed connection to broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:03:48 client/metadata no available broker to send metadata request to
[sarama ] 2020/09/22 09:03:48 client/brokers resurrecting 1 dead seed brokers
[sarama ] 2020/09/22 09:03:48 client/metadata retrying after 250ms... (1 attempts remaining)
[sarama ] 2020/09/22 09:03:49 client/metadata fetching metadata for all topics from broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:03:49 Connected to broker at ecsc00a060af.epam.com:9092 (unregistered)
[sarama ] 2020/09/22 09:04:19 client/metadata got error from broker -1 while fetching metadata: read tcp 10.0.2.100:44244->10.6.194.60:9092: i/o timeout
[sarama ] 2020/09/22 09:04:19 Closed connection to broker ecsc00a060af.epam.com:9092
[sarama ] 2020/09/22 09:04:19 client/metadata no available broker to send metadata request to
[sarama ] 2020/09/22 09:04:19 client/brokers resurrecting 1 dead seed brokers
[sarama ] 2020/09/22 09:04:19 Closing Client
failed to create cluster admin: kafka: client has run out of available brokers to talk to (Is your cluster reachable?)
Config is very simple (sed to show just non-comments and non-empty lines):
$ sed 's/#.*$//;/^$/d' .kafkactl.yaml
contexts:
sandbox:
name: sandbox
brokers:
- ecsc00a060af.epam.com:9092
current-context: sandbox
And according to wireshark dump TCP session kafkactl send metadata request and can connect to the server, but newer receive response:
Where I am wrong?
Error when trying to reset consumer group offset
I'm trying to reset the connect consumer group offset and get an error.
Command: kafkactl reset cgo --oldest -t users connect-cluster
Error: kafka server: The provider group protocol type is incompatible with the other members.
Support for SSL
I've been using kafkactl setting up a kafka cluster and finding it very useful, thanks a bunch.
Now I want to put this thing in production, and that includes only listening on SSL.
But unless I'm missing something, it seems kafkactl will only connect to brokers over plaintext.
Is SSL support something that's on the roadmap? The todo lists Add Security Features but that could be many things.
Thank you for considering adding SSL support.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.