tzachshabtay / krowser Goto Github PK
View Code? Open in Web Editor NEWWeb UI to browse kafka, schema registry and kafka-connect
License: MIT License
Web UI to browse kafka, schema registry and kafka-connect
License: MIT License
We currently destructure the value to separate columns, but the key itself can also be a object, we can try destructuring it as well.
We want to show the consumer groups which subscribe to a specific topic.
Waiting for support from kafkajs: tulios/kafkajs#755
Allow configurable timeouts for the various server requests.
Add helm charts for k8s support
We currently use the open source ag-grid community version.
The enterprise version has more features which we can benefit from (enum filters, column selectionn, groupings and aggregations, and the ability to expand a row so we can show the json view inside the grid).
Need to check if ag-grid has a license which allows open source projects to use the enterprise edition. Otherwise, perhaps we can add it as a feature flag (the user will need to enter their ag-grid license key which will unlock the extra features).
In the 2 message search views (for a single topic and for multiple topics) we want to be able to search by time range (as an alternative to searching by offsets).
Waiting for support from kafkajs: tulios/kafkajs#604 or tulios/kafkajs#525
Currently krowser is read-only.
We can allow editing (under a feature flag, some people will probably just want a read-only experience) various things in kafka and schema-registry (like CRUD operations for topics, partition assignments, subjects, configs, modifying consumer group offsets).
Each one of those should be in its own feature flag for maximum flexibility.
waiting for upstream support: ovotech/castle#74
warning in browser, solution seems to be upgrading to react 17.0.2: https://stackoverflow.com/questions/66489286/sharedarraybuffer-will-require-cross-origin-isolation-as-of-m91-around-may-2021
In the multi-topics search view, need to replace the simple select to be able to type the topics.
We can try material-ui autocomplete instead of the select (we do want to keep the select/clear all checkbox though).
Add optional authentication and RBAC authorization (via SAML?).
You currently only see 3 digits, ideally it should auto-fit the contents
Currently we force people to have schema registry. We should have a feature flag for it to disable all features schema registry related, for those who use kafka without it.
We're currently using datetime-local
for our datetime pickers which is not supported on firefox.
Material-ui are building a datetime picker we should use instead once it's hits stable: https://material-ui.com/components/pickers/
An automated UI test for krowser would be helpful.
Can evaluate playright
Add the ability to extend krowser with custom decoders.
Have an auto-refresh toggle button for all the queries (thinking of a similar experience as kibana/grafana).
Feature request:
Show the "last offset" as of a timestamp. Current method requires start/end time and loads everything inbetween those times - have to incrementally move the window to find "last as of X" with this current state and loading everything can be unpredictable.
Uncaught (in promise) SyntaxError: Unexpected token < in JSON at position 0 async function (async) (anonymous) @ fetcher.tsx:410 callCallback @ react-dom.development.js:12490 commitUpdateQueue @ react-dom.development.js:12511 commitLifeCycles @ react-dom.development.js:19858 commitLayoutEffects @ react-dom.development.js:22803 callCallback @ react-dom.development.js:188 invokeGuardedCallbackDev @ react-dom.development.js:237 invokeGuardedCallback @ react-dom.development.js:292 commitRootImpl @ react-dom.development.js:22541 unstable_runWithPriority @ scheduler.development.js:653 runWithPriority$1 @ react-dom.development.js:11039 commitRoot @ react-dom.development.js:22381 finishSyncRender @ react-dom.development.js:21807 performSyncWorkOnRoot @ react-dom.development.js:21793 (anonymous) @ react-dom.development.js:11089 unstable_runWithPriority @ scheduler.development.js:653 runWithPriority$1 @ react-dom.development.js:11039 flushSyncCallbackQueueImpl @ react-dom.development.js:11084 flushSyncCallbackQueue @ react-dom.development.js:11072 discreteUpdates$1 @ react-dom.development.js:21893 discreteUpdates @ react-dom.development.js:806 dispatchDiscreteEvent @ react-dom.development.js:4168
async function (async) | |
-- | -- | --
| (anonymous) | @ | fetcher.tsx:410
| callCallback | @ | react-dom.development.js:12490
| commitUpdateQueue | @ | react-dom.development.js:12511
| commitLifeCycles | @ | react-dom.development.js:19858
| commitLayoutEffects | @ | react-dom.development.js:22803
| callCallback | @ | react-dom.development.js:188
| invokeGuardedCallbackDev | @ | react-dom.development.js:237
| invokeGuardedCallback | @ | react-dom.development.js:292
| commitRootImpl | @ | react-dom.development.js:22541
| unstable_runWithPriority | @ | scheduler.development.js:653
| runWithPriority$1 | @ | react-dom.development.js:11039
| commitRoot | @ | react-dom.development.js:22381
| finishSyncRender | @ | react-dom.development.js:21807
| performSyncWorkOnRoot | @ | react-dom.development.js:21793
| (anonymous) | @ | react-dom.development.js:11089
| unstable_runWithPriority | @ | scheduler.development.js:653
| runWithPriority$1 | @ | react-dom.development.js:11039
| flushSyncCallbackQueueImpl | @ | react-dom.development.js:11084
| flushSyncCallbackQueue | @ | react-dom.development.js:11072
| discreteUpdates$1 | @ | react-dom.development.js:21893
| discreteUpdates | @ | react-dom.development.js:806
| dispatchDiscreteEvent | @ | react-dom.development.js:4168
This happened after receiving "bad gateway" from the server in
krowser/src/client/kafka/messages/fetcher.tsx
Line 302 in bf86ffb
krowser/src/client/kafka/messages/fetcher.tsx
Line 412 in bf86ffb
Support KSQL view(s)
This is shown in console every time we clear the limit or offset text fields
react_devtools_backend.js:2557 Warning: Received NaN for the `value` attribute. If this is expected, cast the value to a string.
in input (created by ForwardRef(InputBase))
in div (created by ForwardRef(InputBase))
in ForwardRef(InputBase) (created by WithStyles(ForwardRef(InputBase)))
in WithStyles(ForwardRef(InputBase)) (created by ForwardRef(Input))
in ForwardRef(Input) (created by WithStyles(ForwardRef(Input)))
in WithStyles(ForwardRef(Input)) (created by ForwardRef(TextField))
in div (created by ForwardRef(FormControl))
in ForwardRef(FormControl) (created by WithStyles(ForwardRef(FormControl)))
in WithStyles(ForwardRef(FormControl)) (created by ForwardRef(TextField))
in ForwardRef(TextField) (created by WithStyles(ForwardRef(TextField)))
in WithStyles(ForwardRef(TextField)) (created by InputField)
in InputField (created by Fetcher)
in div (created by Fetcher)
in div (created by ForwardRef(Toolbar))
in ForwardRef(Toolbar) (created by WithStyles(ForwardRef(Toolbar)))
in WithStyles(ForwardRef(Toolbar)) (created by Fetcher)
in Fetcher (created by SingleTopicInput)
in SingleTopicInput (created by Messages)
in Messages (created by Context.Consumer)
in Route (created by App)
in div (created by App)
in Router (created by BrowserRouter)
in BrowserRouter (created by App)
in ThemeProvider
in Unknown (created by App)
in App
Partitions are not necessarily sorted when returned from kafka api, causing the offset to be wrong in the UI.
If kafka is restarted when krowser is running, krowser doesn't know how to recover, and has the following error:
Failed to fetch topic mytopic. Error: {"name":"KafkaJSNumberOfRetriesExceeded","retriable":false,"originalError":{"name":"KafkaJSProtocolError","retriable":true,"type":"UNKNOWN_TOPIC_OR_PARTITION","code":3},"retryCount":5,"retryTime":14262}
Hi! I tried launch krowser with confluent kafka as broker.
My compose file:
version: '2.1'
services:
zookeeper:
image: confluentinc/cp-zookeeper:6.2.0
hostname: zookeeper
container_name: zookeeper
ports:
- "2181:2181"
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
broker:
image: confluentinc/cp-server:6.2.0
hostname: broker
container_name: broker
depends_on:
- zookeeper
ports:
- "9092:9092"
- "9101:9101"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_CONFLUENT_LICENSE_TOPIC_REPLICATION_FACTOR: 1
KAFKA_CONFLUENT_BALANCER_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_JMX_PORT: 9101
KAFKA_JMX_HOSTNAME: localhost
KAFKA_CONFLUENT_SCHEMA_REGISTRY_URL: http://schema-registry:8081
CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: broker:29092
CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1
CONFLUENT_METRICS_ENABLE: 'true'
schema-registry:
image: confluentinc/cp-schema-registry:6.2.0
hostname: schema-registry
container_name: schema-registry
depends_on:
- broker
ports:
- "8081:8081"
environment:
SCHEMA_REGISTRY_HOST_NAME: schema-registry
SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: 'broker:29092'
SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:8081
krowser:
image: tzachs/krowser:latest
restart: unless-stopped
hostname: krowser
ports:
- "9999:9999"
environment:
KAFKA_URL: broker:29092
SCHEMA_REGISTRY_URL: http://localhost:8081
I confused with error:
krowser_1 | {"level":"ERROR","timestamp":"2021-07-05T20:32:25.000Z","logger":"kafkajs","message":"[BrokerPool] Failed to connect to seed broker, trying another broker from the list: Connection error: connect ECONNREFUSED 127.0.0.1:9092","retryCount":4,"retryTime":4080}
Could you explain what the right way to start work with confluent kafka as broker?
Best regards!
In the schemas view, we don't currently destructure union or arrays of objects into separate columns.
Warning: Can't perform a React state update on an unmounted component. This is a no-op, but it indicates a memory leak in your application. To fix, cancel all subscriptions and asynchronous tasks in the componentWillUnmount method.
in Topics (created by Context.Consumer)
overrideMethod | @ | react_devtools_backend.js:2557
-- | -- | --
| printWarning | @ | react-dom.development.js:88
| error | @ | react-dom.development.js:60
| warnAboutUpdateOnUnmountedFiberInDEV | @ | react-dom.development.js:23161
| scheduleUpdateOnFiber | @ | react-dom.development.js:21169
| enqueueForceUpdate | @ | react-dom.development.js:12678
| Component.forceUpdate | @ | react.development.js:490
| fetchTopic | @ | topics.tsx:120
| async function (async) | |
| fetchTopic | @ | topics.tsx:92
| componentDidMount | @ | topics.tsx:87
| async function (async) | |
| componentDidMount | @ | topics.tsx:76
| commitLifeCycles | @ | react-dom.development.js:19814
| commitLayoutEffects | @ | react-dom.development.js:22803
| callCallback | @ | react-dom.development.js:188
| invokeGuardedCallbackDev | @ | react-dom.development.js:237
| invokeGuardedCallback | @ | react-dom.development.js:292
| commitRootImpl | @ | react-dom.development.js:22541
| unstable_runWithPriority | @ | scheduler.development.js:653
| runWithPriority$1 | @ | react-dom.development.js:11039
| commitRoot | @ | react-dom.development.js:22381
| finishSyncRender | @ | react-dom.development.js:21807
| performSyncWorkOnRoot | @ | react-dom.development.js:21793
| (anonymous) | @ | react-dom.development.js:11089
| unstable_runWithPriority | @ | scheduler.development.js:653
| runWithPriority$1 | @ | react-dom.development.js:11039
| flushSyncCallbackQueueImpl | @ | react-dom.development.js:11084
| flushSyncCallbackQueue | @ | react-dom.development.js:11072
| scheduleUpdateOnFiber | @ | react-dom.development.js:21199
| enqueueSetState | @ | react-dom.development.js:12639
| Component.setState | @ | react.development.js:471
| (anonymous) | @ | Router.js:34
| listener | @ | history.js:155
| (anonymous) | @ | history.js:173
| notifyListeners | @ | history.js:172
| setState | @ | history.js:288
| (anonymous) | @ | history.js:311
| confirmTransitionTo | @ | history.js:145
| handlePop | @ | history.js:309
| handlePopState | @ | history.js:294
This happens when navigating from the topics window when it's still loading (but can happen in other windows as well), need to add "is mounted" checks and return, or a cancellable promise: https://reactjs.org/blog/2015/12/16/ismounted-antipattern.html.
Currently it's selectable but copying doesn't work.
In addition to avro, schema registry also support protobuf, we can add support for automatic detection and decoding of protobuf as well.
We currently save theme selection to local storage but that's it.
We should consider also saving grid preferences for different queries (like sorting and column order).
To allow sharing urls
We currently decode avro values, but keys might also be avro encoded, we want to try decode them as well.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.