Problem with :db/valueType :db.type/string about datalevin HOT 12 CLOSED

It seems to me that it's an emacs configuration problem. See Emacs Wiki. It shows an example with exactly the same symptom as you described, and it can be fixed by an emacs command.

I use Spacemacs, so everything is configured correctly already and I have never had this problem.

from datalevin.

Here a minimal example for reproducing this behavior:

(require '[datalevin.core :as d])

;; -- buggy (with type specified in schema) --
(def schema {:bug {:db/valueType :db.type/string}})
(def conn (d/get-conn "bug.db" schema))

(d/transact! conn [{:bug "Ümläüt"}])

(d/q '[:find (pull ?e [*])
       :where
       [?e :bug]]
     @conn)
;; => ([{:db/id 1, :bug "��ml����t"}])

(d/close conn)

;; -- correct (without type specified in schema) --
(def conn (d/get-conn "correct.db" {}))

(d/transact! conn [{:bug "Ümläüt"}])

(d/q '[:find (pull ?e [*])
       :where
       [?e :bug]]
     @conn)
;; => ([{:db/id 1, :bug "Ümläüt"}])

Please let me know if I can help more.

• Axel

from datalevin.

Oh, version used is 0.4.18 btw. :-)

from datalevin.

I cannot reproduce this error with 0.4.19 on Linux, nor on Mac. Please let me know more information on your environment, e.g. which OS, Java version, editor, terminal, etc.

from datalevin.

Hi, thanks for answering so quickly.

I'm on a Mac (Mac OS 11.2.2).

java --version
openjdk 11.0.10 2021-01-19
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.10+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.10+9, mixed mode)

Clojure REPL via Emacs/Cider. deps.edn project.

I'll try with 0.4.19.

from datalevin.

Same problem here with version 0.4.19. Unfortunately.
As if a 16 bit encoding is being used when the string type has been specified. Hm, strange.

from datalevin.

Can you try 0.4.20? Thanks

from datalevin.

I tried now outside of Emacs/Cider. In Terminal in the same project folder. REPL started via clj. The "bug" was gone.
Sorry for making waves here and thanks again for your help.
The problem seems to be something with Emacs/Cider here. Will try in VS Code/Calva.

from datalevin.

I will also try 0.4.20 in Emacs/Cider now.
...
0.4.20 changed the behavior. In the Cider REPL now the result of the find pull is
([{:db/id 1, :bug "?ml??t"}])

No longer 16 bit strangeness but unknown chars or something like that. Will open the exact same db in Terminal via clj to see, if it's only a display issue in Emacs/Cider.
...
No, same (faulty) result in Terminal when reading the db which has been created inside Emacs/Cider.

=> 0.4.20 changed something here but we are not completely there. :-)

Maybe there is one little change more ... that fixes this issue (even for Emacs/Cider)? I will look at the diff between 0.4.19 and 0.4.20. The area in which the change happened seems to be relevant.

from datalevin.

All looks good in VSCode/Calva.
Don't know if this issue should bother any longer ... what do you think?
Again, thanks a lot.

from datalevin.

I thought about this a little more, relying on JVM default encoding looks a little dangerous, for it means that the database files may become un-transferable from one OS/locale to another. For example, the default encoding of JVM on Mac/Linux is UTF-8, but the default on Windows is not. We would like to have more control on encoding. So I have decided to use UTF-8 everywhere in our code base, will release 0.4.22 to address this.

Please test 0.4.22 to see if your problem is addressed.

from datalevin.

That kind of rubustness sounds really good.

Just tested version 0.4.22 with Emacs/Cider and it worked flawlessly.
Thanks once again.

from datalevin.