Giter VIP home page Giter VIP logo

kafka_consumer_tool's Introduction

kafka consumer tool

Kafka consumer tool

This tool can be used to set offsets, get offsets and list consumers in a consumer group. Right now it works for Kafka 0.9 onwards and with the new consumer. I am planning to add support for all 0.8 version and old consumer as well.

Installation

You need to have golang installed to install the tool. I haven't uploaded any prebuilt binary for any platoform. Run the below command:

$ go get github.com/nipuntalukdar/kafka_consumer_tool

Running the tool

Example to set offsets of topic (example input json topic_offsets.json file is included along with the source):

$ kafka_consumer_tool -consumergroup newgroup -inputjson topic_offsets.json -brokers 127.0.0.1:9092 --command commitoffsets

Detailed usage shown below:

Usage of kafka_consumer_tool:
  -brokers string
        Bootstrap brokers comma separated list (default "127.0.0.1:9092")
  -command string
        command to execute, valid commands are: commitoffsets, getoffsets, listconsumers (default "getoffsets")
  -consumergroup string
        consumer group name (default "testgroup")
  -inputjson string
        JSON file containing the input details, needed for getoffsets and commitoffsets (default "topic_offsets.json")
  -offsetRetentionTime int
        Offset retention time for commitoffsets in milliseconds (default 3155760000000)

The schema for Input json (needed for commitoffsets and getoffsets) is as shown below:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "items": {
    "type": "object",
    "properties": {
      "offsets": {
        "items": {
          "items": {
            "type": "integer"
          },
          "type": "array",
          "maxItems": 2,
          "minItems": 2
        },
        "type": "array"
      },
      "topic": {
        "type": "string",
        "minLength": 1,
        "maxLength": 1024
      }
    },
    "additionalProperties": false,
    "required": [
      "topic",
      "offsets"
    ]
  },
  "type": "array"
}

kafka_consumer_tool's People

Contributors

nipuntalukdar avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.