datomic / fressian Goto Github PK

I'm the author of a library that provides Java bindings for Clojure's data modeling and serialization capabilities. I've been working on adding Fressian support and I've noticed that, other than the StreamingWriter class, the Fressian library is not at all self-documenting. There is some out-of-band documentation on GitHub, but many crucial concepts (e.g. InheritanceLookup) I had to figure out by reading code. It's true that data.fressian is fairly self-documenting, but I'm targeting Java developers who would be coding directly against the Java reference implementation and not the Clojure bindings.

It appears that the implementation of the UTF-8 encoding wasn't written to properly support UTF-16 surrogate pairs. Was this intentional? If you provide a surrogate pair to the api now you end up with 6 bytes instead of the expected 4 defined by UTF-8.

It will decode correctly using the corresponding decoding implementation but standard implementations of the UTF-8 decoders will not be able to properly decode it.

in the pom.xml there's a relatively ancient clojure dep.

I could not find dependency information in README or anywhere else. This is a major usability problem for beginners. Please add one, for example.

Right now the writer allows writing raw byte arrays, but then on reader there are no public methods for retrieving that arrays back.

If you write an open list, end it, and then write a footer the validation logic for the number of bytes fails. The length reported in the footer and the calculated length are off by one. Here is a example that produces the error

(require '[clojure.data.fressian :as fress])
(import '[org.fressian.impl ByteBufferInputStream BytesOutputStream InheritanceLookup])
(let [my-os (BytesOutputStream.)
      fw (fress/create-writer my-os) ]
  (do
   (fress/begin-open-list fw)
   (fress/end-list fw)
   (fress/write-footer fw)
   (fress/write-object fw nil)
   (let [ my-is (fress/to-input-stream (.internalBuffer my-os))
         fr (fress/create-reader my-is)]
     [(fress/read-object fr) (fress/read-object fr)])))

I believe the cause is in the beginOpenList method of FressianWriter

public Writer beginOpenList() throws IOException {
        if (0 != rawOut.getBytesWritten())
            throw new IllegalStateException("openList must be called from the top level, outside any footer context.");
        writeCode(Codes.BEGIN_OPEN_LIST);
        rawOut.reset();
        return this;
    }

The byte code for the open list is written, then the stream is reset making the reported number of bytes written zero, not one.