tirkarthi / clj-foundationdb Goto Github PK

When you do (get-val tr "test") it's the same as (get-val tr ["test"]). I'm pretty sure that's what you intended, but from the point of view of somebody who is just learning how to use FoundationDB, it's really surprising.

I'd say it would be nice to convert only sequences to tuples, and leave strings as just bytes.

Currently clear-all and get-all use xFF as the end value for Range constructor as per https://stackoverflow.com/a/21421524/2610955 . The stack overflow comment has a Java translation but for some reason it doesn't seem to work though the Python version works fine. I don't know if I am mistaking the escaping rules in Java or packing it the wrong way to Range constructor. Fix these and add valid tests.

Python version :

>>> import fdb
>>> fdb.api_version(510)
>>> db = fdb.open()
>>> db.get_range(b'', b'\xFF')
[b'bar': b'1', b'bar1': b'1', b'bar2': b'1', b'car': b'1', b'foo': b'1']
>>> db.clear_range(b'', b'\xFF')
>>> db.get_range(b'', b'\xFF')
[]

Since the library uses JAR that is not present in the Maven it's not published yet to Clojars. Once it's published to Maven clean up the repository and installation instructions for release.

Upstream issue : apple/foundationdb#219

Implement automatic docs update when a commit is made. Since docs are hosted using GH pages Travis gives a mechanism through which a GitHub token can be generated and the encrypted code can be used as an environment variable to make a commit.

Reference :

• https://github.com/drdoctr/doctr
• https://gist.github.com/domenic/ec8b0fc8ab45f39403dd

Currently Travis fails due to the default cluster missing for FoundationDB. Look into this or other sample projects where this issue is handled and resolve this.

Log : https://travis-ci.org/tirkarthi/clj-foundationdb/builds/370659607

FoundationDB provides watches through which you can see the changes made to a key and so on. There are no examples of this in the Java API and look into how it can be implemented.

Java docs : https://apple.github.io/foundationdb/javadoc/index.html
Examples with Python : https://apple.github.io/foundationdb/developer-guide.html#watches

Describe the bug

Collections passed to set-val as the value return only the first element on trying to retrieve the key

To Reproduce

(let [fd    (. FDB selectAPIVersion 510)
      key   "foo"
      value [1 2 3]]
  (with-open  [db (.open fd)]
    (tr! db
         (set-val tr key value)
         (get-val tr key))))

1 

Expected behavior

It should ideally return [1 2 3]

Additional context

This is caused due to the fact we encode values as tuples themselves and then expect them to be single elements.

(defn get-val
  "
  Get the value for the collection of keys as tuple
  "
  [tr key]
  (let [key   (key->packed-tuple key)]
    (if-let [value @(.get tr key)]
      (.get (Tuple/fromBytes value) 0)))) ;; This should be .getItems

As mentioned above we should be using .getItems but this will return everything as a vector and hence 1 will become [1]. This will break tests and cause unexpected behavior where we can't know if 1 is set or [1] is set. Look into this.

One possible solution will be to pass in some optional key like :seq so that on true we use .getItems and on false we return single elements.

When we pass limit value to lass-less-than or last-less-than-equal it seems to go to the value and then move forward from there for the given offset. E.g. bar1 gives bar but passing limit of 2 gets back again to bar1 and limit of 3 moves from there and so. This seems little strange since the vice-versa greater than work fine. Look into the usage of offsets and the behavior in Python. Add more tests towards this area.

(let [fd (. FDB selectAPIVersion 510)
      keys [["bar"] ["bar1"] ["bar2"] ["car"]]
      value "1"]
  (with-open [db (.open fd)]
    (tr! db (set-keys tr keys value))))

nil

(let [fd (. FDB selectAPIVersion 510)]
  (with-open [db (.open fd)]
    (tr! db (last-less-than "bar1"))))

[[["bar"] "1"]]

(let [fd (. FDB selectAPIVersion 510)]
  (with-open [db (.open fd)]
    (tr! db (last-less-than tr "bar1" 2))))

[[["bar"] "1"] [["bar1"] "1"]]  ;; Seems to go to last less than and then add from there

(let [fd (. FDB selectAPIVersion 510)]
  (with-open [db (.open fd)]
    (tr! db (last-less-than tr "bar1" 3))))

[[["bar"] "1"] [["bar1"] "1"] [["bar2"] "1"]]

(let [fd (. FDB selectAPIVersion 510)]
  (with-open [db (.open fd)]
    (tr! db (last-less-or-equal tr "bar1"))))

[["bar1" "1"]]

(let [fd (. FDB selectAPIVersion 510)]
  (with-open [db (.open fd)]
    (tr! db (last-less-or-equal tr "bar1" 3))))

[[["bar1"] "1"] [["bar2"] "1"] [["car"] "1"]]

(let [fd (. FDB selectAPIVersion 510)]
  (with-open [db (.open fd)]
    (tr! db (last-less-or-equal tr "bar1" 10))))

[[["bar1"] "1"] [["bar2"] "1"] [["car"] "1"] [["foo"] 1]]

(let [fd (. FDB selectAPIVersion 510)]
  (with-open [db (.open fd)]
    (tr! db (last-less-than  tr "bar1" 10))))

[[["bar"] "1"] [["bar1"] "1"] [["bar2"] "1"] [["car"] "1"] [["foo"] 1]]

Some pointers : https://github.com/apple/foundationdb/blob/e0c8175f3ccad92c582a3e70e9bcae58fff53633/bindings/java/src/test/com/apple/foundationdb/test/StackTester.java#L688

Is your feature request related to a problem? Please describe.

Provide function to return metrics

Describe the solution you'd like

GET \xff\xff/status/json returns the metrics for the cluster as JSON

Additional context

Ref : https://github.com/leoluk/fdb_exporter/blob/eba4a65606ff3ce66b7f560d5ab63eeb1cb7f637/fdb_exporter.py#L22

Implement subspace API and check for compatibility with the current API. Better to start a new namespace since core is getting longer.

Python API : https://apple.github.io/foundationdb/api-python.html#subspaces

Reference : https://apple.github.io/foundationdb/javadoc/com/apple/foundationdb/subspace/Subspace.html

Using docs folder in master branch was easy to maintain but over the period the commit history is being polluted with a lot of update docs commits with 8 out of 29 commits at the time of opening the issue. I think it will be better to migrate docs to gh-pages to keep the master branch clean.