backend uses OpenApi for API, the idea is to use the same OpenApi config 'kafka-ui/kafka-ui-contract/src/main/resources/swagger/kafka-ui-api.yaml' for client requests

https://github.com/swagger-api/swagger-js

Time to retain is not working properly
Max size on disk is not working properly

Check other configs

I've tried to deploy in Cloud along with MSK, based on https://github.com/provectus/quickstart-provectus-streaming-data-platform-kafka
The KAFKA_CLUSTERS_0_ZOOKEEPER param passed as string, the example
"z-1.mskcluster-test1.q75ugu.c3.kafka.us-west-2.amazonaws.com:2181,
z-2.mskcluster-test1.q75ugu.c3.kafka.us-west-2.amazonaws.com:2181,
z-3.mskcluster-test1.q75ugu.c3.kafka.us-west-2.amazonaws.com:2181"

UI itself is available, and represent passed cluster with Brokers, Topics list, topic messages and settings.

Under the Brokers tab, "ZOOKEEPER STATUS" is present, but show that zookeeper offline.

The zookeeper itself is accessible from Kafka-UI Fargate task, the Kafka-UI log proff this. In the log there "connection established" messages with each Zookeeper nodes.

I've tried to find the reason for this by enabling debug in log4j2, but didn't found any valuable information.

The example of Kafka-UI log below.

kafka-ui-cloudlog.txt

Generate cluster ID from cluster name

Frontend sends get topic before post action.
There is no need to make additional get, info could be retrieved from post response

Field Time to retain data in topic creation should be customizable. Now it's only allow to choose from enum

Change sorting of topic messages - sort by offset in descending order (latest messages first).

• Removed unused mock server
• Updated Readme

• Provide a list of available topic parameters with default values
• Update the topic creation endpoint to use list of custom parameters
• Validate custom parameters

Acceptance criteria:

• consumer groups component added
  • the user can see list of groups with number of consumers & number of topics
  • the user can filter groups by name
• consumer group component added
  • the user can see list of consumers with topic, partition, lag, messages behind, vcurrent offset, end offset

blocked by #9

Update In Sync Replicas & Out of Sync Replicas on broker's page to use ISR & OSR params from cluster's payload

blocked by: #5

If topic message content is a valid JSON it should be displayed as a tree with expandable nodes.

Acceptance criteria:

• New endpoint to get the list of topic messages added.
• implemented the ability to filter topic messages by:
  • partition
  • offset
  • timestamp
• Endpoint should be pageable
• Add schema registry url config
• Add schema registry to docker compose
• Provision one topic with messages schema in schema registry
• Trait default content as a text
• topics could be configured:
  • text
  • json
  • avro (schema registry)

at least for all clusters made up from kafka-clusters-only.yaml

https://github.com/provectus/kafka-ui/blob/master/kafka-ui-api/src/main/java/com/provectus/kafka/ui/kafka/KafkaService.java#L132

Add ISR & OSR params to cluster's payload

Implement CRUD for cluster tokens.
Should be under a new tab in cluster menu below "Consumers"

After topic edit, internal store is not updated with applied configs.

Fix zookeeper status cheking

Please add ESlint with strict config. Ask @aMoRRoMa about best config

Acceptance criteria:

1. Provide a list of consumer groups connected to the cluster
2. In consumer group, provide a list of consumer id with topic, partition, lag, messages behind, current offset, end offset

Add min/max offset to topic partition entity. Request each time.

We need to add:

• clear instructions how to install and run the app
  • e.g: brew cask install java doesn't seem to work anymore
• info about supported Node versions

Add ability to view list of consumers for topic.

Add red/green status light somewhere around cluster name in sidebar

https://github.com/provectus/kafka-ui/blob/master/kafka-ui-api/src/main/java/com/provectus/kafka/ui/rest/MetricsRestController.java#L71

Some requests with TIMESTAMP fails, e.g.

/api/clusters/local/topics/users/messages?limit=100&seekType=TIMESTAMP&seekTo=0::1594120200949&seekTo=1::1594120200949

returns

message: "java.lang.NullPointerException"
path: "/api/clusters/local/topics/users/messages"
requestId: "71f158dc"
status: 500
timestamp: 1594120799702```

Acceptance criteria

1. New component added
2. The user can filter messages by keyword & partition
3. The user can render messages from offset or timestamp

blocked by: #10

1. Run npm run lint from /kafka-ui-react-app
2. Specify a correct type instead of any for all cases from the list

src/components/Topics/Details/Messages/Messages.tsx
  Line 41:29:   Unexpected any. Specify a different type  @typescript-eslint/no-explicit-any
  Line 76:13:   Unexpected any. Specify a different type  @typescript-eslint/no-explicit-any
  Line 101:52:  Unexpected any. Specify a different type  @typescript-eslint/no-explicit-any
  Line 181:43:  Unexpected any. Specify a different type  @typescript-eslint/no-explicit-any
  Line 230:53:  Unexpected any. Specify a different type  @typescript-eslint/no-explicit-any

src/redux/store/configureStore/dev.ts
  Line 9:16:  Unexpected any. Specify a different type  @typescript-eslint/no-explicit-any

case: When u create a topic with an existing name, there will be response with error from backend, but front will freeze, and will not show any error

expected: backend errors must be represented in UI

AC

The user is able to:

• add multiple custom parameters to topic creation form
• select available custom topic parameter from list
• update value of added parameter
• remove added parameter
• review validation error & update custom params

blocked by: #7

1. The endpoint to get metrics list
2. The endpoint to get metrics values
3. Endpoint with the list of most important metrics

Add ability to manage ACL for cluster

https://github.com/provectus/kafka-ui/blob/master/kafka-ui-api/src/main/java/com/provectus/kafka/ui/cluster/model/InternalClusterMetrics.java#L23

1. Get Total cluster space consumption
2. Get Broker space consumption
3. Get Topic space consumption (include & exclude replicas)

Filters should be applied only after submit button

1. Add prometheus node exporter to one cluster (for all nodes)
2. Add config to resolve node exporter addresses per broker
3. Config to read metrics from node exporter or jmx
4. Implement metrics reading from node exporter

Now we are waiting for n messages or timeout. Let's check the max partition offset before starting a reading.

1. Google
2. GitHub - available orgs
3. GitLab - available groups
4. Custom

• Research: determine set of metrics for visualisation
• Implement metrics visualisation

Frontend should get cluster metrics from cluster metrics endpoint (production / consumption)

/api/clusters/{clusterName}/metrics

{topic}/settings page should be editable with possibility to override settings

1. Add query parameter js-filter with java script predicate function
2. Add Nashorn dependency
3. Filter messages before string inclusion using this function and record
4. Pass variables to function:
   4.1 key (key object)
   4.2 value (content object)
   4.3 headers
   4.4 offset
   4.5 partition

UPD: design for the frontend part is already done on figma

• Support message preview (partial selection of json fields)
• Support searching by message headers (requested in #780)