Giter VIP home page Giter VIP logo

pysnowflake's Introduction

pysnowflake

pysnowflake is a Python implementation of Twitter's snowflake service - https://github.com/twitter/snowflake

I implemented it in pure Python using the same thrift interfaces because I really dislike the whole overhead the Java Runtime brings with it. Not to mention the fact that its really hard to run it on small servers with little RAM because you have to fine tune it to the right memory amounts.

With Python its easier and quicker to setup.

Due to various reasons this implementation does not reach the performance indicated in the original Snowflake implementation, however it is good enough in most cases and can be combined with the help of a software load balancer such as HAProxy to run multiple processes to get higher performance.

You can checkout a Python client implementation I did which I use with this server at: https://github.com/erans/pysnowflakeclient

Installation

  • Make sure you have the Thrift Python libraries (0.61 and above would be great)
  • Run the service

Usage

usage: pysnowflake.py [-h] [--host HOST] [--port PORT] [--log_level LOG_LEVEL] [--verbose] worker_id data_center_id

Python based Snowflake server

positional arguments: worker_id Worker ID data_center_id Data Center ID

optional arguments: -h, --help show this help message and exit --host HOST Host address (default 127.0.0.1) --port PORT port (default 30303) --log_level LOG_LEVEL Log level (default: INFO). Values: ERROR,WARN,INFO,DEBUG --verbose Be verbose!

Running it

./pysnowflake worker_id data_center_id (i.e. ./pysnowflake 1 1)

Requirements

Issues

Please report any issues via github issues

pysnowflake's People

Contributors

erans avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.