Note: SciRuby are taking over maintenance for this library: github.com/SciRuby/packable
If you need to do read and write binary data, there is of course Array::pack
and String::unpack
. The packable library makes (un)packing nicer, smarter and more powerful. In case you are wondering why on earth someone would want to do serious (un)packing when YAML & XML are built-in: I wrote this library to read and write FLV files…
Strings, integers & floats have long forms instead of the cryptic letter notation. For example:
["answer", 42].pack("C3n")
can be written as:
["answer", 42].pack({:bytes => 3}, {:bytes => 2, :endian => :big})
This can look a bit too verbose, so let’s introduce shortcuts right away:
Most commonly used options have shortcuts and you can define your own. For example:
:unsigned_long <===> {:bytes => 4, :signed => false, :endian => :big}
IO classes (File & StringIO) can use (un)packing routines. For example:
signature, block_len, temperature = my_file >> [String, :bytes=>3] >> Integer >> :float
The method each
also accepts packing options:
StringIO.new("\000\001\000\002\000\003").each(:short).to_a ===> [1,2,3]
It’s easy to make you own classes (un)packable. All the previous goodies are thus available: File.open(“great_flick.flv”) do |f| head = f.read(FLV::Header) f.each(FLV::Tag) do |tag| # do something meaningful with each tag… end end
It’s also easy to define special shortcuts that will call blocks to (un)pack any class. As an example, this could be useful to add special packing features to String (without monkey patching String::pack).
First, ensure that you’re running at least RubyGems 1.2 (check gem --version
if you’re not sure – to update: sudo gem update --system
).
Add GitHub to your gem sources (if you haven’t already):
sudo gem sources -a http://gems.github.com
Get the gem:
sudo gem install marcandre-packable
That’s it! Simply require 'packable'
in your code to use it.
Designed to work with ruby 1.8 & 1.9.
The library was designed to be backward compatible, so the usual packing and unpacking methods still work as before. All packable objects can also be packed directly (no need to use an array). For example:
42.pack("n") ===> "\000*"
In a similar fashion, unpacking can done using class methods:
Integer.unpack("\000*", "n") ===> 42
Although the standard string formats can still be used, it is possible to pass a list of options (see example in feature summary). These are the options for core types:
bytes
-
Number of bytes (default is 4) to use.
endian
-
Either
:big
(or:network
, default),:little
or:native
. signed
-
Either
true
(default) orfalse
. This will make a difference only when unpacking.
precision
-
Either
:single
(default) or:double
. endian
-
Either
:big
(or:network
, default),:little
or:native
.
bytes
-
Total length (default is the full length)
fill
-
The string to use for filling when packing a string shorter than the specified bytes option. Default is a space.
repeat
-
This option can be used (when packing only) to repeat the current option. A value of
:all
will mean for all remaining elements of the array.
When unpacking, it is necessary to specify the class in addition to any option, like so:
"AB".unpack(Integer, :bytes => 2, :endian => :big, :signed => false) ===> 0x3132
It’s easy to add shortcuts for easier (un)packing:
String.packers.set :flv_signature, :bytes => 3, :fill => “FLV”
"x".pack(:flv_signature) ===> "xFL"
Two shortcut names have special meanings: default
and merge_all
. default
specifies the options to use when nothing is specified, while merge_all
will be merged with all options. For example:
String.packers do |p| p.set :merge_all, :fill => "*" # Unless explicitly specified, :fill will now be "*" p.set :default, :bytes => 8 # If no option is given, this will act as default end "ab".pack ===> "ab******" "ab".pack(:bytes=>4) ===> "ab**" "ab".pack(:fill => "!") ===> "ab" # Not "ab!!"
A shortcut can refer to another shortcut, as so:
String.packers do |p|
p.set :creator, :bytes => 4 p.set :app_type, :creator end
"hello".pack(:app_type) ===> "hell"
The following shortcuts and defaults are built-in the library:
:merge_all => :bytes=>4, :signed=>true, :endian=>:big :default => :long :long => {} :short => :bytes=>2 :byte => :bytes=>1 :unsigned_long => :bytes=>4, :signed=>false :unsigned_short => :bytes=>2, :signed=>false
:merge_all => :precision => :single, :endian => :big :default => :float :double => :precision => :double :float => {}
:merge_all => :fill => " "
All IO objects (in particular files) can deal with packing easily. These examples will all return an array with 3 elements (a string, an integer and another string):
io >> :flv_signature >> Integer >> [String, {:bytes => 8}] io.read(:flv_signature, Integer, [String, {:bytes => 8}]) io.read(:flv_signature, Integer, String, {:bytes => 8}) [io.read(:flv_signature), io.read(Integer), io.read(String, :bytes => 8)]
In a similar fashion, these have the same effect although the return value is different
io << "x".pack(:flv_signature) << 66.pack << "Hello".pack(:bytes => 8) # returns io io << ["x", 66, "Hello"].pack(:flv_signature, {} , {:bytes => 8}) # returns io io.write("x", :flv_signature, 66, "Hello", {:bytes => 8}) # returns the # of bytes written io.packed << ["x",:flv_signature] << 66 << ["Hello", {:bytes => 8}] # returns a "packed io"
The last example shows how io.packed
returns a special IO object (a packing IO) that will pack arguments before writing it. This is to insure compatibility with the usual behavior of IO objects: io << 66 ==> appends “66” io.packed << 66 ==> appends “000000000B”
We “cheated” in the previous example; instead of writing io.packed.write(...)
we used the shorter form. This works because we’re passing more than one argument; for only one argument we must call io.packed.write(66)
less the usual write
method is called.
Since the standard library desn’t define the >>
operator for IO objects, we are free to use either io.packed
or io
directly. Note that reading one value only will return that value directly, not an array containing that value:
io.read(Integer) ===> 42, not [42] io.read(Integer,Integer) ===> [42,43] io << Integer ===> [42]
Including the mixin Packable
will make a class (un)packable. Packable relies on write_packed
and unpacking on read_packed
. For example:
class MyHeader < Struct.new(:signature, :nb_blocks) include Packable def write_packed(packedio, options) packedio << [signature, {:bytes=>3}] << [nb_blocks, :short] end def self.read_packed(packedio, options) h = MyHeader.new h.signature, h.nb_blocks = packedio >> [String, {:bytes => 3}] >> :short h end end
We used the argument name packedio
to remind us that these are packed IO objects, i.e. they will write their arguments after packing them instead of converting them to string like normal IO objects. With this definition, MyHeader
can be both packed and unpacked:
h = MyHeader.new("FLV", 65) h.pack ===> "FLV\000A" StringIO.new("FLV\000A") >> Signature ===> [a copy of h]
A default self.read_packed
is provided by the Packable
mixin, which allows you to define read_packed
as an instance method instead of a class method. In that case, read_packed
instance method is called with the same arguments and should modify self
accordingly (instead of returning a new object). It is not necessary to return self
. The previous example can thus be shortened:
class MyHeader #... def read_packed(packedio, options) self.signature, self.nb_blocks = packedio >> [String, {:bytes => 3}] >> :short end end
Instead of writing a full-fledge class, sometimes it can be convenient to define a sort of wrapper we’ll call filter. Here’s an example:
String.packers.set :length_encoded do |packer| packer.write { |packedio| packedio << length << self } packer.read { |packedio| packedio.read(packedio.read(Integer)) } end
“hello!”.pack(:length_encoded) ===> “000000000006hello!” [“this”, “is”, “great!”].pack(**3).unpack(**3) ===> [“this”, “is”, “great!”]
Note that the write
block will be executed as an instance method (which is why we could use length
& self
), while read
is a normal block that must return the newly read object.
A final note to say that packers are inherited in some way. For instance one could define a filter for all objects:
Object.packers.set :with_class do |packer| packer.write { |io| io << [self.class.name, :length_encoded] << self } packer.read do |io| klass = eval(io.read(:length_encoded)) io.read(klass) end end [42, MyHeader.new("Wow", 1)].pack(:with_class, :with_class).unpack(:with_class, :with_class) ===> [42, MyHeader.new("Wow", 1)]
packable is licensed under the terms of the MIT License, see the included LICENSE file.
- Author
-
Marc-André Lafortune