jimpil / duratom Goto Github PK

What do you think about moving the necessary namespaces to cljc and implementing a backend on top of browser localStorage (https://developer.mozilla.org/en-US/docs/Web/API/Web_Storage_API/Using_the_Web_Storage_API)? Then we would have client side duratom.

Is it a silly idea?

Would it be possible to provide an option to have reset! and swap! commit the write to the backend synchronously before returning? Sometimes it's important that the writer know that the change was successfully persisted.

Hi, nice project, many thanks!

I'm using it for the first time, and in my project already has next.jdbc as a dependency. Question: would a PR replacing clojure.java.jdbc for next.jdbc would be well received?

Thanks again!

i'm basically getting an error telling me that a my_duratom.tmp -> my_duratom is throwing a java.nio.file.NoSuchFileException

I get this error with Java 8. I guess the DirectByteArrayOutputStream Java class has been compiled with Java8.

Syntax error macroexpanding at (utils.clj:1:1).
Execution error (UnsupportedClassVersionError) at java.lang.ClassLoader/defineClass1 (ClassLoader.java:-2).
dbaos/DirectByteArrayOutputStream has been compiled by a more recent version of the Java Runtime (class file version 53.0), this version of the Java Runtime only recognizes class file versions up to 52.0

This morning I started seeing this:

WARNING: identity already refers to: #'clojure.core/identity in namespace: duratom.utils, being replaced by: #'duratom.utils/identity

Guessing it's caused by this line:

So maybe these:

Could instead say:

(juxt #(first %&) f)

😄

Hope you're well. As ever, thanks for duratom, we still have zillions of nippy duratoms with gigabytes of stuff in them and we love it.

Hi! I hope you're well. I still use this library every day and it is great!

I noticed the version is up to "0.4.1", and I haven't upgraded in a while. What are the recent changes? Are they things I'll want to upgrade for? Have there been any breaking api changes?

I poked around the CHANGELOG.md, but it looks like the default lein template one.

Regardless, thanks again.

Yes, I know it's big. I have a big hash map. I'm moving to a database backend, it's probably time, but it's worth knowing this is an issue.

I'd love to help debug to fix this if I can.

I'm taking about this place https://github.com/jimpil/duratom/blob/master/src/clojure/duratom/utils.clj#L25

Given https://github.com/clojure/clojure/blob/ee3553362de9bc3bfd18d4b0b3381e3483c2a34c/src/clj/clojure/core.clj#L3672L3674 and according to https://clojuredocs.org/clojure.core/*print-dup* I understand that print-dup is preferred way to serialise anything in order to restore it later, and print-method is used for console printing, just human representation etc..

Meaningful differences:

user=> ((juxt identity (comp meta read-string)) (binding [*print-dup* true] (pr-str (with-meta {:data 1} {:meta 1}))))
["^#=(clojure.lang.PersistentArrayMap/create {:meta 1}) #=(clojure.lang.PersistentArrayMap/create {:data 1})" {:meta 1}]
user=> ((juxt identity (comp meta read-string)) (binding [*print-dup* false] (pr-str (with-meta {:data 1} {:meta 1}))))
["{:data 1}" nil]

So duratom loses meta info by default. On the other hand print-dup string should be read-string not a edn/read-string.

I suspect introducing this will break compatibility with written files, and I understand that :rw gives an easy option to configure it, just wonder why such default? Only because of sticking with edn?

Maybe warn users about this as well / add example in README how to serialise via print-dup/restore with read-string?

https://github.com/ptaoussanis/nippy is a wicked serialization library that we use all over the place for storing edn.

The main trade-offs of nippy vs pr-str are:

• pr-str is plain-text and nippy is binary byte blob (opaque)
• nippy is smaller on disk (sometimes ridiculously so, ~20x?!)
• nippy reads and writes faster

Now that I am storing non-trivial data in .edn duratoms, it occured to me that nippy might make an interesting backend for this lib.

Any thoughts? Any interest in a PR?

Nippy has low level and high level apis, as such the data saved by freeze can't actually be read by thaw-from-in! because freeze/thaw is a high level api supporting encryption, compression and including the nippy file header, where thaw-from-in! and freeze-from-in! are low level and don't support those, so when it is trying to read the type id and instead finds the first letter of the NPY header, it complains that it doesn't recognise the file.

I also have an issue in that I get a 404 from s3 if the key doesn't exist yet and I don't know what to put in it to manually create a blank nippy file as a starting point.

If the user has print-length set then any lengthy lists etc will get cut off with [1 2 3 ...] which not only loses the data but isn't valid edn and won't read back in. Need to wrap calls to pr-str with (binding [*print-length* nil] (pr-str whatever))

Nice idea! wanted to try duratom, but cannot build it into my project because of this compilation error below. I'm using Clojure 1.9:

Clojure.lang.Compiler$CompilerException: java.lang.RuntimeException: No such var: sql/with-db-transaction, compiling:(duratom/utils.clj:71:3)
             java.lang.RuntimeException: No such var: sql/with-db-transaction

Hi Dimitrios,
thanks for duratom!

After upgrading to version 0.4.0 I get the following error:

No such var: aws/get-object-metadata, compiling:(duratom/utils.clj:147:37)

I just use the file backend and thus didn't add the optional requirement amazonica. It almost seems that something which needs AWS stuff slipped into the new version.

This is a feature request (or proposal for one).

I've a use case which fits duratom well: caching and operating on a value in memory with the benefit of durability in the event of process restart.

In my case the initial-value is very expensive to calculate, so I want to avoid calculating it unless it is necessary. I can achieve this now with passing nil as the value of :init, then wrapping the duratom in a check to populate the value if necessary.

I thought it would be a nice addition to the library however if :init was optionally a function that would be invoked by duratom to create the initial value.

Hi, not an actual user, so don't prioritise this.

It occurred to me when browsing though, that it's not clear to me whether this is suitable for use from multiple servers simultaneously or not. For example if host A and B both call (swap! a inc) simultaneously, is it guaranteed that the number will increase by 2, or will it potentially increase only by 1?