cmiles74 / bencode Goto Github PK

There's a suggestion to allow converting structures such as {:key :val} or [:done :error] to bencode. Currently it complains that it cannot convert keywords. Other implementations I saw (particularly inspired by clojure implementation) convert keywords to strings in this case.

I can see a reasoning that it is not allowed in that the type information is lost during this conversion and you can't guarantee that

(= str (decode (encode str)))

So maybe one option is to hide this behind a flag such as strict-conversion. Or maybe it's not that important and always convert keywords to strings. Converting to strings is definitely more convenient for the casual use case, but there's also a comment on why bencode will never support floating point numbers which says differently https://lists.ibiblio.org/pipermail/bittorrent/2004-August/001040.html.

Implementations:

• https://github.com/pyrmont/bencodobi - janet implementation which does that
• https://nrepl.org/nrepl/0.8/building_clients.html#implementing-sideloading - clojure implementation also does that, can be seen from examples. It actually goes even further and converts symbols to strings as well https://github.com/nrepl/nrepl/blob/4eb9ad189f97dc88c70b4163a6c2ca327f9a95fc/test/clojure/nrepl/bencode_test.clj#L147-L152

If you are okay with the idea, I can suggest and provide a pull request for the following:

• Add strict-conversion flag to all the public write functions, true by default so the default follows specification (probably keyword-style as in #3 if that gets accepted)
• When this is false, convert keywords and symbols to strings

I think this restriction can be lifted for 0 since bencode spec allows it. I will prepare a pull request if you are okay with it.

Bencode spec https://www.bittorrent.org/beps/bep_0003.html#bencoding, also some resources and implementations use zero length strings, for example https://github.com/sanel/monroe

Hi, I really enjoy your library! I have a question about the choice of mutable data structures for representation, why was the decision to choose them instead of immutable ones?

My main use case is that buffers @"abc" can't be used in comparisons easily, meaning that (= @"a" @"a") prints false. This makes it impossible to use in expressions such as match and probably others.

I use this function to change everything to immutable structures currently but I wonder if there are any ideas of changing the default one to immutable too:

(defn deep-immutabilize
  "Turn every mutable structure to immutable one."
  [value]
  (defn immutabilize [v]
    (match (type v)
      :struct (deep-immutabilize v)
      :table (deep-immutabilize v)
      :buffer (string v)
      :array (map immutabilize v)
      :tuple (map immutabilize v)
      _ v))
  (def t (type value))
  (cond
    (or (= :table t) (= :struct t))
    (struct ;(mapcat
              (fn [x]
                (let [first (0 x) second (1 x)]
                  [(keyword first) (deep-immutabilize second)]))
              (pairs value)))
    (or (= :array t) (= :tuple t))
    (tuple ;(map deep-immutabilize value))
    (immutabilize value)))

If #2 gets merged, there will be already 3 optional parameters to the read functions. They all create some kind of specialized behavior. I usually see such cases utilizing the keyword-style arguments https://janet-lang.org/docs/functions.html.

I would suggest to use this style for all public read functions: read, read-buffer and read-stream. The reason is just a more convenient API for the library. If you think this is good, I can prepare a pull request.

In read-dictionary and read-list the code uses match-byte to check that the value is properly terminated:

But match-byte also advances to the next byte in the stream. If this is a network stream we possibly wait forever to get a next byte and never return the fully parsed value.

A quick fix is to just use peek-byte when we are parsing a top-level list or dictionary. I implemented this in
felixr@c4c59e0

It looks like it is working, but I admittedly have not tried hard to make it nice and went with the first thing that came to my mind.