Giter VIP home page Giter VIP logo

gafka's Introduction

gafka Build Status

                   __   _            
                  / _| | |           
   __ _    __ _  | |_  | | __   __ _ 
  / _` |  / _` | |  _| | |/ /  / _` |
 | (_| | | (_| | | |   |   <  | (_| |
  \__, |  \__,_| |_|   |_|\_\  \__,_|
   __/ |                             
  |___/                              

A full ecosystem for kafka/redis/PubSub/ElasticSearch/Zookeeper/haproxy.

Hope it can help you.

Components

  • ehaproxy

    Elastic haproxy that sits in front of kateway.

  • kateway

    A fully-managed real-time secure and reliable RESTful Cloud Pub/Sub streaming message/job service.

  • actord

    kateway job scheduler and webhook dispatcher.

  • gk

    Unified multi-datacenter multi-cluster kafka swiss-knife management console.

  • zk

    A handy zookeeper CLI that supports recursive operation without any dependency.

  • es

    ElasticSearch console.

  • kguard

    Kafka clusters body guard that emits health info to InfluxDB and exports key warnings to zabbix for alerting.

Install

export PATH=$PATH:$GOPATH/bin

#========================================
# install go-bindata and go annotations
#========================================
go install github.com/jteeuwen/go-bindata/go-bindata
go install github.com/funkygao/goannotation

#========================================
# install gafka
#========================================
go get github.com/funkygao/gafka
cd $GOPATH/src/github.com/funkygao/gafka
./build.sh -a # build all components

#========================================
# try the gafka command 'gk'
#========================================
gk -h

Status

Currently gafka manages:

  • 4 data centers
  • 50+ kafka clusters
  • 100+ kafka brokers
  • 500+ kafka topics
  • 2000+ kafka partitions
  • 10Billion messages per day
  • peak load
    • 1Million message per second
    • 8TB transfered per hour
  • 5TB redis

gafka's People

Contributors

funkygao avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.