A Clojure interface to Google's protocol buffers

Contents

• About
• Changes in this Fork
• Using Leiningen
• Getting Started
• Usage
• Documentation
• Release Notes
• Donating

This project provides a Clojure interface to Google's protocol buffers. Protocol buffers can be used to communicate with other languages over the network, and they are WAY faster to serialize and deserialize than standard Clojure objects.

Changes in Protocol Buffer's "syntax3" require some changes in the library to correct 'bug's:

1. mapdef.clj was failing at line 39 due to changes in the protobuf Java implementation. Specifically, messages are Descriptors$FileDescriptor, not the Descriptors$Descriptor required.

To run all the tests, type:

lein test-all

To work at the repl, type:

lein repl

There are some "Rich Comments" for repl'ing in:

• syntax_3_test.clj
• core.clj
• mapdef.clj

Add the dependency to your project.clj:

Then, given a project with the following in resources/proto/your/namespace/person.proto:

package your.namespace.person;

option java_outer_classname = "Example";

message Person {
  required int32  id    = 1;
  required string name  = 2;
  optional string email = 3;
  repeated string likes = 4;
}

you can compile the proto using the protobuf compiler and include the resulting .java code in your project:

protoc \
  -I=/usr/include \
  -I=/usr/local/include \
  -I=resources/proto \
  --java_out=$OUT_DIR \
  resources/proto/your/namespace/*.proto

Note that, at this point, the files are .java source files, not .class files; as such, you will still need to compile them.

We've found a clean way to do this (and how we set up the tests) is to:

• put these .java files in an isolated directory
• add that directory to a :java-source-paths entry in the project.clj
• place that in an appropriate project.clj profile

Now you can use the protocol buffer in Clojure:

(require '[protobuf.core :as protobuf])
(import '(your.namespace.person Example$Person))

(def alice (protobuf/create Example$Person
                            {:id 108
                             :name "Alice"
                             :email "[email protected]"}))

Make some changes to the data and serialize to bytes:

(def b (-> alice
           (assoc :name "Alice B. Carol")
           (assoc :likes ["climbing" "running" "jumping"])
           (protobuf/->bytes)))

Round-trip the bytes back to a probuf object:

(protobuf/bytes-> alice b)

Which gives us:

{:id 108,
 :name "Alice B. Carol",
 :email "[email protected]",
 :likes ["climbing" "running" "jumping"]}

The data stored in the instance is immutable just like other clojure objects. It is similar to a struct-map, except that you cannot insert fields that aren't specified in the .proto file.

(For instance, if you do a round trip with the data like we did above, but use :dislikes -- not in the protobuf definition -- instead of :likes, converting from bytes back to the protobuf instance will result in the :dislikes key and associated value being dropped.)

The above usage is a quick taste; for more examples as well as the current and previous reference documentation, visit the Clojure protobuf documentation. These docs include the following:

• Basic usage example
• A Clojure port of Google's Java protobuf tutorial
• An example for working with extensions
• Clojure protobuf API Reference
• Marginalia docs
• Documentation for the two Java classes (see the "javadoc" link there)

A note on the history: This project picked up the Clojure protobuf code base from the ghaskins fork; its last significant update was 2 years prior. The work at that point had been given the version "3.4.0-2-SNAPSHOT", tracking the Protocol Buffer release of 3.4.0. We created a branch and tag for that release with no changes other than the org/artifact id.

At the request of a user, a donation account for project development has been set up on Liberapay here:

• https://liberapay.com/clojusc-protobuf/donate

You can learn more about Liberapay on its Wikipedia entry or on the service's "About" page.