A Haskell library for fast, memory-efficient decoding of JSON documents using the simdjson C++ library
Home Page: https://hackage.haskell.org/package/hermes-json
License: MIT License
repl> newtype Person = Person Int deriving Show
repl> personDecoder = withObject $ \obj -> Person <$> atKey "lol" int obj
repl> decodeEither personDecoder "\x00"
Left (SIMDException (HError {path = "", errorMsg = "A JSON document made of a scalar (number, Boolean, null or string) is treated as a value. Use get_bool(), get_double(), etc. on the document instead. ", docLocation = "", docDebug = "json_iterator [ depth : 1, structural : '\NUL/\DEL\NUL\NULO\aB\NUL\NUL\NULA\nB\NUL\NUL\NULh(\DEL\NUL\NUL\SOH\NUL\NUL\NUL"}))
repl> decodeEither personDecoder "\x00"
Left (SIMDException (HError {path = "", errorMsg = "A JSON document made of a scalar (number, Boolean, null or string) is treated as a value. Use get_bool(), get_double(), etc. on the document instead. ", docLocation = "", docDebug = "json_iterator [ depth : 1, structural : '\NULM\ETXB\NUL\NUL\NUL`(\DEL\NUL\NUL9M\ETXB\NUL\NUL\NUL7M\ETXB\NUL\NUL\NUL\EMe^\EOT"}))
repl> decodeEither personDecoder "\x00"
Left (SIMDException (HError {path = "", errorMsg = "A JSON document made of a scalar (number, Boolean, null or string) is treated as a value. Use get_bool(), get_double(), etc. on the document instead. ", docLocation = "", docDebug = "json_iterator [ depth : 1, structural : '\NUL\STX\tB\NUL\NUL\NULy\ETXB\NUL\NUL\NULJ~\ETXB\NUL\NUL\NULh(\DEL\NUL\NUL\SOH\NUL\NUL\NUL"}))
Scroll to the end of the line to see some garbage data.
The fact that the same input bytestring returns unequal errors breaks referential transparency, which users expect.
This also makes it harder to use Hermes in property testing.
Here is an expression that I would expect to be True:
repl> fromLeft undefined (decodeEither personDecoder "\x00") == fromLeft undefined (decodeEither personDecoder "\x00")
False
Where are the instructions on how to build hermes? Tried the following on MacOS Monterey 12.6
👍
% cabal --version
cabal-install version 3.8.1.0
% cabal update
Downloading the latest package list from hackage.haskell.org
Package list of hackage.haskell.org has been updated.
The index-state is set to 2022-10-10T16:14:10Z.
To revert to previous state run:
cabal v2-update 'hackage.haskell.org,2022-01-03T18:47:46Z'
gregory@our-imac hermes % cabal build
Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] trying: hermes-json-0.2.0.0 (user goal)
[__1] next goal: base (dependency of hermes-json)
[__1] rejecting: base-4.17.0.0/installed-4.17.0.0 (conflict: hermes-json =>
base>=4.13 && <4.17)
[__1] skipping: base-4.17.0.0 (has the same characteristics that caused the
previous version to fail: excluded by constraint '>=4.13 && <4.17' from
'hermes-json')
[__1] rejecting: base-4.16.3.0, base-4.16.2.0, base-4.16.1.0, base-4.16.0.0,
base-4.15.1.0, base-4.15.0.0, base-4.14.3.0, base-4.14.2.0, base-4.14.1.0,
base-4.14.0.0, base-4.13.0.0, base-4.12.0.0, base-4.11.1.0, base-4.11.0.0,
base-4.10.1.0, base-4.10.0.0, base-4.9.1.0, base-4.9.0.0, base-4.8.2.0,
base-4.8.1.0, base-4.8.0.0, base-4.7.0.2, base-4.7.0.1, base-4.7.0.0,
base-4.6.0.1, base-4.6.0.0, base-4.5.1.0, base-4.5.0.0, base-4.4.1.0,
base-4.4.0.0, base-4.3.1.0, base-4.3.0.0, base-4.2.0.2, base-4.2.0.1,
base-4.2.0.0, base-4.1.0.0, base-4.0.0.0, base-3.0.3.2, base-3.0.3.1
(constraint from non-upgradeable package requires installed instance)
[__1] fail (backjumping, conflict set: base, hermes-json)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, hermes-json
Similar to haskell/text#455.
Ok, maybe you don't even wanna support this. But it would be nice to specify if this is on purpose.
Aeson 2 can roundtrip plus/minus infinity and NaN just fine:
repl> Data.Aeson.decode @Double $ encode @Double ((1)/0)
Just Infinity
repl> Data.Aeson.decode @Double $ encode @Double ((-1)/0)
Just (-Infinity)
repl> Data.Aeson.decode @Double $ encode @Double ((0)/0)
Just NaN
But Hermes can't decode these encodings:
repl> newtype Person = Person Double deriving (Eq, Show)
repl> personDecoder = Data.Hermes.withObject $ \obj -> Person <$> atKey "lol" double obj
repl> Data.Hermes.decodeEither personDecoder $ "{\"lol\": " <> (toStrict $ encode @Double ((1)/0)) <> "}"
Left (SIMDException (HError {path = "/lol", errorMsg = "Error while getting value of type double. The JSON element does not have the requested type.", docLocation = "\"+inf\"}", docDebug = "json_iterator [ depth : 2, structural : '\"', offset : 8', error : No error ]"}))
repl> Data.Hermes.decodeEither personDecoder $ "{\"lol\": " <> (toStrict $ encode @Double ((-1)/0)) <> "}"
Left (SIMDException (HError {path = "/lol", errorMsg = "Error while getting value of type double. The JSON element does not have the requested type.", docLocation = "\"-inf\"}", docDebug = "json_iterator [ depth : 2, structural : '\"', offset : 8', error : No error ]"}))
repl> Data.Hermes.decodeEither personDecoder $ "{\"lol\": " <> (toStrict $ encode @Double ((0)/0)) <> "}"
Left (SIMDException (HError {path = "/lol", errorMsg = "Error while getting value of type double. The JSON element does not have the requested type.", docLocation = "null}", docDebug = "json_iterator [ depth : 2, structural : 'n', offset : 8', error : No error ]"}))
Given that people are likely to use this library as a replacement for Aeson, I think it would be useful to point out differences like this.
*Data.Hermes> newtype Person = Person Int deriving Show
*Data.Hermes>
*Data.Hermes> personDecoder = withObject $ \obj -> Person <$> atKey "lol" int obj
*Data.Hermes> decodeEither personDecoder "{}"
Left (SIMDException (HError {path = "/lol", errorMsg = "The JSON field referenced does not exist in this object.", docLocation = "/\DEL", docDebug = "json_iterator [ depth : 0, structural : 'n', offset : 2', error : No error ]"}))
What does the "/\DEL" mean here? I am worried that it is accessing something out of bounds. I would expect the location to be /.
So I tried running it a few times and something is definitely wrong here:
*Data.Hermes> decodeEither personDecoder "{}"
*** Exception: Cannot decode byte '\xb8': Data.Text.Internal.Encoding.decodeUtf8: Invalid UTF-8 stream
*Data.Hermes> decodeEither personDecoder "{}"
cabal: repl failed for hermes-json-0.1.0.0. The build process segfaulted (i.e.
SIGSEGV).
This is on Debian Bullseye and ghc 8.10.7 from ghcup and commit
commit a3fe3eb9d8da07295d7c7ac3030dc5daaf89f27f (HEAD -> master, origin/master, origin/HEAD)
Author: Josh Miller <[email protected]>
Date: Tue Nov 2 17:18:48 2021 -0500
use CStringLen for JSON pointer as well
Decoder provides a MonadIO instance, which means a decoder can launch nukes. decodeEither centralizes the unsafePerformIO because it is convenient to use IO internally for exceptions and the unliftio machinery.
So if we lose MonadIO then we lose MonadUnliftIO, and we'll have to refactor how we handle error reporting. I think using a continuation-style like how attoparsec does it could be the better approach. All FFI operations will then need to use unsafePerformIO.
Alternatively, we can maybe get away with defining our own versions of lifting/unlifting IO and never export them.
I just found out about the cffi flag for Aeson. It would be nice to know whether the benchmarks in the readme were performed with or without that flag set.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.