Giter VIP home page Giter VIP logo

avro-util's Introduction

Avro-Util

Github Build Status Latest Release

A collection of utilities and libraries to allow java projects to better work with avro.

How To Use

Artifacts are published to artifactory. To use them (say in a gradle build) you'd need to add that repository to your build.gradle, and then add a dependency on the module(s) you wish to use:

repositories {
   maven {
      url  "https://linkedin.jfrog.io/artifactory/avro-util/"
   }
}
...
dependencies {
   compile "com.linkedin.avroutil1:helper-all:<latest version>"
}

Background

Apache Avro is a widely used serialization format.

Unfortunately it is not always possible to write java code that "just works" across multiple versions of avro, as there have been breaking changes to both the API and wire format. This project provides utility code to enable java developers to write code that is compatible across a wide range of avro versions

Furthermore, this project enables Avro-based projects to achieve better performance.

Modules

The following modules are available in this project.

helper

This module provides utility functions which, when coded against, ensure compatibility with every Avro supported version. This is achieved by offering a single API which, under the hood, delegates to the Avro version which is detected at runtime.

avro-codegen

This module provides code generation capabilities which allows an auto-generated SpecificRecord class to work with all supported versions of Avro.

avro-fastserde

This is a fork of https://github.com/RTBHOUSE/avro-fastserde

In a nut shell, avro-fastserde enables faster Avro de/serialization by doing runtime code-generation to provide a faster decoder/encoder implementation.

As part of this fork, there are compatibility improvements to ensure that the decoding semantics are the same as that of regular Avro (in particular around the handling of String types in modern Avro). Moreover, this fork enables partial compatibility with older versions of Avro. Finally, this fork provides supports for object re-use, a standard Avro deserialization feature, as well as some other garbage-collection optimizations which have proven useful at LinkedIn.

Supported versions of Avro

The helper module supports avro 1.4 - 1.11 inclusive, fastserde included.

avro-util's People

Contributors

radai-rosenblatt avatar felixgv avatar gaojieliu avatar srramach avatar guawang avatar majisourav99 avatar flowenol avatar volauvent avatar yuzheng21 avatar majisourav avatar neel24 avatar kcepmada avatar li-ukumar avatar lincong avatar idanz avatar ahmedahamid avatar abhisheknath2011 avatar cbrentharris avatar dg-builder avatar sixpluszero avatar kehuum avatar maciejkowalczyk avatar pravk03 avatar sutambe avatar huangminchn avatar wxn225 avatar xinxinzhenhuai avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.