besok / jsonpath-rust Goto Github PK
View Code? Open in Web Editor NEWSupport for json-path in Rust
Home Page: https://crates.io/crates/jsonpath-rust
License: MIT License
Support for json-path in Rust
Home Page: https://crates.io/crates/jsonpath-rust
License: MIT License
example:
[?(!(...))]
I'm trying this in online json-path evaluators and both i tried working. Maybe i'm doing something wrong? Basically need to express the following: "select all nodes with child node X"
The following example does not work correctly:
Json:
[{"verb": "TEST"}]
Jsonpath:
$.[?(@.verb == 'TEST')].length()
Result of this query will be Null
, while correct result should be 1 as one item was found.
It looks like this happens because if length of result is 1 we can not determine if we should return length of the object of length of the result array: https://github.com/besok/jsonpath-rust/blob/master/src/path/top.rs#L99
Hi,
Thanks for your work :)
To the best of my knowledge, python does not have a well maintained JSONPath library. Maybe it could be a good idea to create python bindings so that this library could be used from python.
Let me start off by saying that I really like this library! It's been very helpful in the week or so since I started using it!
But there are a couple of tasks I was trying to do that I don't think are currently supported, but could be…
The first one is to search for all the fields of a specific type, and then show a sibling element, like components.[?(@.componentType == \"date-input\")].^.name
(if ^
was the parent, and there was a name beside the element).
The second one was to search for items that had been translated with something like $.fields.[?(@.~ != \"en-US\")].*
It looks like JSONPath_Plus could support both of those, and I was wondering if that's something you'd be open to implementing?
Have you seen the crate:json_dotpath?
This isn’t implementing the extensive search functionality that your JsonPathFinder
can perform, but it might provide you with some ideas.
Maybe you could implement a trait over Value itself (taking either an &str or perhaps even a conversion trait (call it C) of sorts)? Then your user could simply call value.find(&str) or value::.find(C) and get a Result back?
That would avoid you having to expose JsonPathFinder to the user. Similarly, you could then implement two traits, one for finding references and one that returns mutable references?
All the user would need to ‘know’ then to use your crate would be how to construct the query language for finding those references. Like the dot_path crate, you could also utilise Serde’s DeserializedOwned trait to extend this beyond Value to any implementor of that trait. That would then be very flexible indeed. As I said, people may disagree, and I only scanned the code; just a few suggestions really. Hope that helps you along the way!
came from LinkedIn comments from Michael Radley
Hello 👋
I wanted to ask if it's possible to do something following with this crate.
I want to be able to run multiple queries on some JSON to eventually find a node that I'd like to modify/remove.
In the following example, I'm trying to remove the genre of the second book from the list.
use std::str::FromStr;
use serde_json::{Value};
use jsonpath_rust::{JsonPathFinder, JsonPathInst, JsonPathQuery};
fn main() {
let data = r#"
{
"books": [
{
"title": "Title 1",
"details": {
"description": "Description 1",
"genre": "Genre 1"
}
},
{
"title": "Title 2",
"details": {
"description": "Description 2",
"genre": "Genre 2"
}
}
]
}
"#;
let json: Value = serde_json::from_str(data).unwrap();
let books_path = JsonPathInst::from_str("$.books").unwrap();
let finder = JsonPathFinder::new(Box::from(json), Box::new(books_path)); // had to clone due to ownership
let books = finder.find_slice().get(0).unwrap().clone().to_data();
let second_book = books.get(1).unwrap();
let genre_value = second_book.clone().path("$.details.genre").unwrap(); // had to clone due to ownership in path
// remove genre_value
}
Do you have any suggestion how to achieve this?
If I'm not mistaken find
returns new Values and find_slice
return reference to the Value. However none of them return mutable reference, maybe adding it could help with this?
Or maybe once the feature mentioned in #31 is added I could find the paths that are to be removed/modified, clone whole "original" json and modify it using these paths 🤔
If you have any other idea please point me to the right direction 🙏
Thanks!
In the Goessner implementation the Json Path query can return either value or path. The Java implementation that we used to use has this feature too.
Would it be possible to add this feature to this library?
Not sure something like [?(@.category=='fiction' && @.price < 10)]
support?
I am not able to use jsonpath that has bool or null values, even though it is mentioned in README. For example this code based on example:
use serde_json::{json,Value};
use jsonpath_rust::json_path_value;
use jsonpath_rust::JsonPathFinder;
use jsonpath_rust::JsonPathValue;
fn main(){
let finder = JsonPathFinder::from_str(r#"{"first":{"second":[{"active":1},{"passive":1}]}}"#, "$.first.second[?(@.active == true)]").unwrap();
let slice_of_data:Vec<JsonPathValue<Value>> = finder.find_slice();
let js = json!({"active":1});
assert_eq!(slice_of_data, json_path_value![&js,]);
}
It is based on the example. Notice the jsonpath string is: $.first.second[?(@.active == true)]
The error when running is:
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: " --> 1:30\n |\n1 | $.first.second[?(@.active == true)]\n | ^---\n |\n = expected atom"', src/main.rs:7:139
Looks like this case is not added to grammar rules
the lib not support like this
{
"data": [1, 2, 3]
}
$.data.length => [ 3 ]
how can i do?
given json:
[
{"verb": "TEST","a":[1,2,3]},
{"verb": "TEST","a":[1,2,3]},
{"verb": "TEST"}, {"verb": "RUN"}
]
given jpath:
$.[?(@.verb == 'TEST')].a.length()
Result: [2]
Expected result: [3,3]
Given the following input document:
"a": {
"b": {
"c": {
"d": true
}
}
},
The query:
$.a[?(@.b.c.d)]
returns Some
But
$.a[?(@.b.c.d==true)]
returns None
The current interface exposed to applications wants to own both the JSON value and the path. It also allows for updating the path and/or value. I would rather not have to pass ownership of my data to the library and simply borrow the data (both pre-parsed path and JSON value). The foundation for this functionality exists internally through a combination of parse_json_path
, JsonPath
, json_path_instance
, and PathInstance
.
By making those public on my own branch, I can get code like this to work:
for item in json_path_instance(&path, &value).find(&value).into_iter() {
// ...
}
But clearly this is not a great user experience since value
is redundantly provided to the library. I suggest a public interface be created using these internals.
I plan to use your library for Grillon and I was wondering if you would agree to add an extra step in your workflow so we can match the tags here with the deployed versions on crates.io? It's currently hard to follow what was integrated in your deployed version.
It's fairly simple in terms of process, but it needs to follow the trunk based development (+ continuous delivery which requires a human action). Each time you send a tag on the main
branch, it triggers a github action to deploy on crates.io. Since tagging is a human action I found it quite good for controlling deployments while automating them.
Also github tag UI comes with release notes that you can use to attach to your tag and automatically link to milestones. It's great to get all the details of a tag 😄 .
Let me know and I can help you to setup this - you will need the crates.IO token to setup as a secret of your git repo.
Json doc:
[ {"verb", "TEST"}, {"verb", "RUN"}]
Json path:
$.[?(@.verb=='Test')]
Returns error:
|
1 | $.[?(@.verb=='Test')]
| ^---
|
= expected string_qt"'
The same expression tested out with online tools like this one: https://jsonpath.com/.
JsonPathFinder seems to be redundant. Probably, better to revise the API a remove it
After reading some bugfixes and playing around with edge cases, I think it would be great if the library could handle an invalid json path as an error instead of returning an empty array. This would avoid some bugs or confusing results I think. The specs recommend to return an empty array or false, but I think this is because of the lack of types at this moment (javascript / php). I think the version by the Internet engineering task force is the most recent one but it's globally the same as the initial version.
An alternative for jsonPath to return false in case of no match may be to return an empty array in future. [This is already done in the above.]
Here "no match" seems to cover both empty results and non-existent paths. It leaves a gray area on the implementation details.
Let's take this Value
as the data source:
let json_obj = json!({
"shop": {
"orders": [
{ "id": 1, "active": true },
{ "id": 2 },
{ "id": 3 },
{ "id": 4, "active": true }
],
"past_orders": [ "AB01", "AB02" ],
"returned_orders": []
}
});
Note: I don't know if the last fixes for length
function have been released or not. So maybe some of the results here are already solved.
We can't differentiate an empty result from an non-existent path. Both json paths will return Array [ Number(0) ]
$.shop.inexistant[*].length()
$.shop.returned_orders[*].length()
I would expect an Err(JsonPathNotFound)
instead for 1. (or similar).
It's more specific to the implementation I'm doing with Grillon but it's also the case for native assertions with the Rust test lib. I can't tell an empty result from a non-inexistent path.
If we keep the initial data source, it's all good. Let's say we want to verify AB01
and AB02
are not in returned_orders
, the assertions passes :
let path = "$.shop.returned_orders[*]";
let val = json_obj.path(path).unwrap();
let should_not_be = json!(["AB01", "AB02"]);
assert_ne!(val, should_not_be);
println!(" val: {val:#?} \n should not be: {should_not_be:#?}");
warning: `jpath-test` (bin "jpath-test") generated 1 warning
Finished dev [unoptimized + debuginfo] target(s) in 0.86s
Running `target/debug/jpath-test`
val: Array []
should not be: Array [
String("AB01"),
String("AB02"),
]
But now let's remove returned_orders
from the data, the result is the following:
warning: `jpath-test` (bin "jpath-test") generated 1 warning
Finished dev [unoptimized + debuginfo] target(s) in 0.90s
Running `target/debug/jpath-test`
val: Array []
should not be: Array [
String("AB01"),
String("AB02"),
]
We observe a false positive. We think there is no issue but in reality the path doesn't exist, the result isn't empty, the data doesn't contain what we're looking at. Instead I would like to handle it as an Error
and produce meaningful information to my users. Like raising a warning, since it's not completely false, you don't have any returned orders but the fact that it's missing from the result shouldn't be ignored.
I don't think I have a way to make distinction right now. I also think it causes some bugs in the interpretation of results in the library and by users. The advantage of a strongly typed language is that we can take advantage on the type system to return meaningful information. I wonder if it would be possible to handle non-existent json paths in the library? Do we have any cases where handling an error could prevent basic functionality from working? If not, can we work on a new version?
I want to use jsonpath-rust
for pathfinding in jsonb project, which is a binary JSON format. But the parser
module is private, can we make the parser
module public so that it can be used independently by downstream projects?
Another issue is that JsonPathIndex
relies on serde_json::Value
, which does not seem necessary. Whether it is possible to use basic types to define the relevant fields?
If the object does not exist length()
still returns 0. 0 is a valid length of an array and this can be a source of bugs. Current behaviour is:
let json: Box<Value> = Box::new(json!([{"verb": "TEST"},{"verb": "TEST"}, {"verb": "RUN"}]));
let path: Box<JsonPathInst> = Box::from(
JsonPathInst::from_str("$.not.exist.length()").expect("the path is correct")
);
let finder = JsonPathFinder::new(json, path);
assert_eq!(finder.find(), json!([0]);
I would like to propose to return Value::Null
in case of unsupported or unknown objects.
I prepared a PR that can fix it: #24
It's helpful for debugging, when I try to create a JsonPathFinder
and it fails I cannot easily debug the error.
Since the calculation happens in memory, there is a pivotal drawback that ensues with the processing of the big files.
But probably before all, the good idea is to attempt to measure it
Heyho,
I think there was no release since the last PR, i would propose a 0.7.0 as there are incompatibilities with 0.6.1 and it could break users code otherwise :)
best regards,
xMAC94x
Hello. It looks like escaping characters in regex is problematic
#[test]
fn regex_filter_test(){
let json: Box<Value> = Box::new(json!({
"author":"d(Rees)",
}));
let path: Box<JsonPathInst> = Box::from(
JsonPathInst::from_str("$.[?(@.author ~= 'd(Rees)')]").expect("the path is correct"),
);
let finder = JsonPathFinder::new(json.clone(), path);
assert_eq!(finder.find_slice(),vec![Slice(&json!({"author":"d(Rees)",}))]);
}
assertion left == right failed left: [NoValue] right: [Slice(Object {"author": String("d(Rees)")})]
If you try to escape with \ it won't work either
'd\(Rees\)' won't compile, expecting an atom
'd\\(Rees\\)' compiles but it will look for \ in value and () is still treated like a capture group
And if you don't escape the () is considered a capture group
Provide an update by path or return a mutable borrow.
Hi!
I'm evaluating integration of jsonpath-rust into my app fhttp. I don't understand why JsonPathFinder.find
returns a Vec<&Value>
instead of just a Value
, which could be a Value::Array
. As someone integrating the lib and not knowing the data or the jsonpath expression used at compile time, i can't know whether i need to unpack the vec or not.
Why was it designed this way?
Hi, I am new to rust and jsonpath, so I am not sure whether this is a bug or not.
When comparing field values using logical expression I need to always check if the field is present or not.
If I use a logical expression with a field that does not exist, jsonpathfinder will return the whole string as a match.
See the example below:
use jsonpath_rust::JsonPathFinder;
use serde_json::json;
// This returns null as expected
fn main() {
let finder = JsonPathFinder::from_str(r#"{"first":{"second":[{"active":1},{"passive":1}]}}"#, "$.first[?(@.does_not_exist && @.does_not_exist >= 1.0)]").unwrap();
let res_js = finder.find();
assert_eq!(res_js, json!(null));
}
// This returns the full string as a match
fn main() {
let finder = JsonPathFinder::from_str(r#"{"first":{"second":[{"active":1},{"passive":1}]}}"#, "$.first[?(@.does_not_exist >= 1.0)]").unwrap();
let res_js = finder.find();
assert_eq!(res_js, json!(null));
}
Here is my Cargo toml
[package]
name = "jsonpath_test"
version = "0.1.0"
edition = "2021"
[dependencies]
jsonpath-rust = "0.3.0"
serde = {version = "1.0.160", features = ["derive"]}
serde_json = "1.0"
Currently parse_json_path
uses unwrap
in a couple of places which is far from good practice.
Shift to normal process handling with Either and introduce an internal parser error
Placeholder to myself or somebody to find some time to refactor the module in the following points
expect
with ok_or(..)
Currently it seems this code is fairly strict in only supporting single quotes, e.g.,
$.key[?(@.size == 'small')].city
I am wondering if we can loosen this restriction and also allow for double quotes, e.g.,
$.key[?(@.size == "small")].city
The reason for this is when using this syntax in a CLI it is best to wrap this in single quotes as $
is a special character for most shells that gets interpolated when using double quotes. So the following would theoretically work without having to escape single quotes...
my_cli --path '$.key[?(@.size == "small")].city' my_data.json
Hey. I'm using this library at work and I have noticed some slowness when using regex, looks like creating a Regex is somehow slow. I have tested the theory using a global map for storing the compiled regex and it improved the speed ~50x. The implementation is not what we want, we should create the regex when creating the jsonpath and use it directly. I'm not sure how we can do that.
Here is the changes I made for testing:
https://github.com/catalinstochita/jsonpath-rust
Let me know if you have an idea of how we can do that. Thank you @besok
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.