Provide "jason_native" about jason HOT 8 CLOSED

The easiest way would be to maintain the parsing stack as a data structure threaded through the functions instead of using recursive calls - this means we can always interrupt, return from the functions, yield and later use that state to resume the parsing from where it was left.

from jason.

I think this is a great idea!

Having multiple Rust binaries for popular targets will be nice as well as using Rustler.

We could also just provide some sort of build step for those comfortable in the Rust ecosystem as well.

Now that serde is on stable we can (with confidence) push out stable/safe native perf for those that chose the alternative

from jason.

Yes, but we can't use serde directly - the problem is that it is not pre-emptive, so we can't use it with regular NIFs. It's true we have dirty schedulers, but I'm pretty sure using them in high concurrency situations for this could be even slower than the pure Elixir version (that statement is not confirmed by any experiments, though).

from jason.

Yea that makes sense. I wonder how this could otherwise be done 🤔

from jason.

@michalmuskala

Yes, but we can't use serde directly - the problem is that it is not pre-emptive

This is a bit unexpected especially as Rust is meant to be highly concurrent and I can't find a proof that serde is not thread safe.

from jason.

@RumataEstor It's not about thread safety. Rust and Elixir does concurrency in different ways, Rust uses OS threads and Elixir uses lightweight processes. When serde runs it will block a scheduler until it is done running so when decoding or encoding large JSON objects you can disrupt the scheduler.

from jason.

I don’t mean it’s not thread safe - it is. Preemptive means we need to be able to pause the execution of the parser/generator at arbitrary moments to yield the scheduler thread to other elixir/erlang processes.

from jason.

After 5 years, I finally got around to this. Check out https://github.com/spawnfest/json_native!

from jason.