java-json-tools / json-schema-core Goto Github PK

Hello, I'm working with some other FOSS projects that are looking at using some of the java-json-tools projects as dependencies.

Where it says that these repos are "dual licensed LGPLv3 or later/ASL 2.0", and also in the LICENSE files that they are "dual-licensed," I've interpreted that as meaning that these repos can be used under either LGPLv3 or ASL 2.0.

Can you confirm that that's the intent here? I've received questions about whether it instead means that downstream projects have to use the code under both LGPLv3 and ASL 2.0, and I don't think that's your intent. Grateful if you can confirm though. Thank you!

My unit-tests use JsonSchemaFactory which indirectly references SchemaVersion. ShemaVersion downloads remote resources. Is there a way I can disable this behavior so that my unit-tests don't need to download remote resources (or require an internet connection)?

https://github.com/fge/json-schema-core/blob/68edbf9ba307e07d15fee0dfea6dc0d47d9155b8/src/main/java/com/github/fge/jsonschema/SchemaVersion.java#L62

URIUtils.SCHEME_CHECKER.check() loads the error message bundle (which spawns a Thread to get the message source for the locale) even if the check would succeed. If this was a Future then it could be lazily loaded only if one of the checks failed.

This was discovered when upgrading json-schema-validator from v1.6.2 to v2.2.10, since the test harness we used for "small" tests inhibits thread creation, and meant that validator tests now must be promoted to "medium".

There are two of them. The first not being that much of a concern, but the second is a huge concern.

The first: duplicate members do not raise an error. That is, with input:

{ "a": "b", "a": "c" }

then whatever is associated with member a is undefined. That is understandable since this is not even valid JSON as per RFC 4627, which says that objects SHOULD NOT have duplicate member names. So, we can live with that. Still, it'd be nice it this could raise an error.

The second is rather embarrassing. This input WILL NOT raise an error:

{} garbage here

It will stop after } and declare the job done!

But the input IS NOT EVEN VALID JSON.

This is quite embarrassing for an API validating JSON. And to work around it, I have to use a JsonParser and build the tree myself :(

Hi,

We saw some crashes on Android 9 device, can you please give some insights/recommendations to the problem?
Thanks,

Library Version: json-schema-core: 1.2.8
json-schema-validator: 2.2.8
Most affected Android Devices: Samsung Galaxy J7
Most affected Android Version: Android 9

Caused by java.util.zip.ZipException invalid stored block lengths java.util.zip.InflaterInputStream.read (InflaterInputStream.java:175) java.net.URL.openStream (URL.java:1072) com.github.fge.jackson.JsonLoader.fromResource (JsonLoader.java:98) com.github.fge.jsonschema.SchemaVersion.<init> (SchemaVersion.java:63) com.github.fge.jsonschema.SchemaVersion.<clinit> (SchemaVersion.java:44) com.github.fge.jsonschema.SchemaVersion.values (SchemaVersion.java:39) com.github.fge.jsonschema.core.load.configuration.LoadingConfigurationBuilder.<init> (LoadingConfigurationBuilder.java:119) com.github.fge.jsonschema.core.load.configuration.LoadingConfiguration.byDefault (LoadingConfiguration.java:151) com.github.fge.jsonschema.main.JsonSchemaFactoryBuilder.<init> (JsonSchemaFactoryBuilder.java:67) com.github.fge.jsonschema.main.JsonSchemaFactory.newBuilder (JsonSchemaFactory.java:123) com.github.fge.jsonschema.main.JsonSchemaFactory.byDefault (JsonSchemaFactory.java:113)

Caused by java.lang.NoClassDefFoundError com.github.fge.jsonschema.SchemaVersion com.github.fge.jsonschema.SchemaVersion.values (SchemaVersion.java:39) com.github.fge.jsonschema.core.load.configuration.LoadingConfigurationBuilder.<init> (LoadingConfigurationBuilder.java:119) com.github.fge.jsonschema.core.load.configuration.LoadingConfiguration.byDefault (LoadingConfiguration.java:151) com.github.fge.jsonschema.main.JsonSchemaFactoryBuilder.<init> (JsonSchemaFactoryBuilder.java:67) com.github.fge.jsonschema.main.JsonSchemaFactory.newBuilder (JsonSchemaFactory.java:123) com.github.fge.jsonschema.main.JsonSchemaFactory.byDefault (JsonSchemaFactory.java:113)

Caused by java.lang.ExceptionInInitializerError com.github.fge.jsonschema.SchemaVersion.<init> (SchemaVersion.java:65) com.github.fge.jsonschema.SchemaVersion.<clinit> (SchemaVersion.java:44) com.github.fge.jsonschema.SchemaVersion.values (SchemaVersion.java:39) com.github.fge.jsonschema.core.load.configuration.LoadingConfigurationBuilder.<init> (LoadingConfigurationBuilder.java:119) com.github.fge.jsonschema.core.load.configuration.LoadingConfiguration.byDefault (LoadingConfiguration.java:151) com.github.fge.jsonschema.main.JsonSchemaFactoryBuilder.<init> (JsonSchemaFactoryBuilder.java:67) com.github.fge.jsonschema.main.JsonSchemaFactory.newBuilder (JsonSchemaFactory.java:123) com.github.fge.jsonschema.main.JsonSchemaFactory.byDefault (JsonSchemaFactory.java:113)

In my json schema, I define some parameters are "required" within a json array. If I miss those "required" parameters in the first element of the json array, the validation logic will detect and report error. If I miss the "required" parameters in the second element or after of json array, the validation logic does not detect the error and report success instead.

Links from json-schema-validator into the core also give 404.

In a static initialization block in RegexECMA262Helper.java the available JavaScript engines are initialized. Javas Nashorn engine is pretty straight forward and can be used easily. That's the primary script engine which is always used if Java 8 - 14 is used.

However the backup engine Rhino is also initialized thus making it impossible to remove the Rhino lib from every application that uses the json-schema-core although it is unused during the whole runtime. 😞

Why not just initialize Rhino if Nashorn is not available?

Hi, this call in BaseSchemaTree:

public final String toString()
    {
        return Objects.toStringHelper(this)
            .add("key", key)
            .add("pointer", pointer)
            .add("URI context", currentRef)
            .toString();
    }

Objects.toStringHelper has been deprecated and removed in later versions of guava. Is there another way to generate this string that doesn't depend on the Objects.toString?

JsonPointer will be migrated into this package.

Goal: make it more self contained, for it to be more easily integrated into jackson-core.

Original issue:

java-json-tools/json-schema-validator#38

TODO:

• de-guavaify;
• split ReferenceToken;
• make it not implement JsonFragment (turn the latter into an interface instead);
• others?

I'm having the same problem as documented in this swagger issue.

It seems clear that the author wants the $ref URIs to be normalized, so I'm very hesitant to call this a bug.

Mainly, I'm curious if there is justification for this requirement. If normalization is required due to an implementation detail, couldn't the implementation just normalize the URI internally?

The README states this packages can be used to generate a JSON Schema from a POJO. Is there any sort of documentation to begin to move around this goal? Or maybe this was already done?

A lot of JSON Schema software only supports version 3, so this is probably the single most important piece of information to put in README.md.

When the DefaultURIDownloader tries to load a schema and got a response 301 Moved Permanently it tries to parse the body content and fails (For example check zalando/zally#1164).

Would be great if the DefaultURIDownloader could handle redirects and instead of failing, try to download schema again.

In order to update the version of Guava, or, in fact mention that we depend on Guava (see issue #49), we'll need to convert uses of Iterators.emptyIterator() to ImmutableSet.<T>of().iterator() per https://guava.dev/releases/19.0/api/docs/com/google/common/collect/Iterators.html#emptyIterator()

Migrating from java-json-tools/json-schema-validator#291.

BaseSchemaTree's constructors do not agree about whether to use JsonPointer.path which will provide the missing node if the path was not found, or JsonPointer.get which will just provide a null node. Since getNode is not @nullable, I think the third constructor is just incorrect.

With Java 15 Nashorn was finally removed from the JDK. But with GraalJS there is some sort of replacement.

If the application runs with the polyglot GraalVM there is out of the box support for GraalJS.
Or GraalJS and its ScriptEngine could be simply added as dependency and ran with the JDK.

Is there any way to validate that the JsonSchema is valid upon loading the JSON. For example, a user uploads a JsonSchema to a registry and the registry validates that the submitted JSON is a valid JsonSchema before persisting?

URIUtils schema URI checker used for URITranslatorConfigurationBuilder.addSchemaRedirect() checks path of URI not ending with slash.

Please verify check and normalization correlation. Or maybe just NPE aid will do the thing.
Thanks

I used the online validator on http://json-schema-validator.herokuapp.com/
This tool successfully validates the schema below with the data below
This is wrong, because the billing_address is a boolean instead of a string.
It looks like the billing_address type is being ignored.

The validator should fail because the type is declared as a string whereas the data contains a boolean.

Schema:

{
  "type": "object",

  "properties": {
    "name": { "type": "string" },
    "credit_card": { "type": "number" }
  },

  "required": ["name"],

  "dependencies": {
    "credit_card": {
      "properties": {
        "billing_address": { "type": "string" }
      },
      "required": ["billing_address"]
    }
  }
}

Data:

{                                     
  "name": "Johan",
  "credit_card_number": 5555555555555555,
  "billing_address": true
}

I have an use case where I need to compare two json schemas and check if the schema A is compatible with schema B are not. To do this, I need to access to the underlying SchemaTree data structures constructed from the raw json schema file. Could you expose the internal schema data structures used in this package ? Or is there some other way I can get hold of the schema tree.

Thanks,
Venkat.

I understand there are differences between the regex in JS and in Java, but it would be good to figure out if they are meaningful in light of json-schema. Loading up a Rhino context to use simple patterns is, well, heavy.

Looking at com.github.fge.jsonschema.report.ProcessingMessage I do not see a way to get access to any of the values you have stored in it's map property. I am particularly interested in the schema pointer an instance pointer. I'd like to be able to tell people that when they have an invalid email address:

"#/email_addresses/0 is an invalid email address"

I see no way to access the map however... Am I missing something, or is this not currently possible?

https://mvnrepository.com/artifact/org.easytesting/fest-assert/1.4 is very old and looks to be unmaintained. The http://fest.easytesting.org site now tries to sell you sesamin (a sesame-seed-based compound). There's no obvious source location, making it difficult to build from source inside Google's monorepo.

Figure out its successor and update tests accordingly.

One hint is from alexruiz/fest-assert-2.x#167 (comment) where one of the main contributors points would-be user to AssertJ.

java.lang.NoSuchFieldError: JAVA_ISO_CONTROL
	at com.github.fge.uritemplate.parse.CharMatchers.<clinit>(CharMatchers.java:40)

https://github.com/fge/uri-template/blob/master/src/main/java/com/github/fge/uritemplate/parse/CharMatchers.java

URIUtils propose a SCHEMAURI_CHECKER.
But if the URI begins with "jar:file:", the argument.getPath() will return a null value then we will get a null pointer exception.

    private static final ArgumentChecker<URI> SCHEMAURI_CHECKER
        = new ArgumentChecker<URI>()
    {
        @Override
        public void check(final URI argument)
        {
            BUNDLE.checkArgumentPrintf(argument.isAbsolute(),
                "uriChecks.notAbsolute", argument);
            final JsonRef ref = JsonRef.fromURI(argument);
            BUNDLE.checkArgumentPrintf(ref.isAbsolute(),
                "uriChecks.notAbsoluteRef", argument);
            BUNDLE.checkArgumentPrintf(!argument.getPath().endsWith("/"),
                "uriChecks.endingSlash", argument);
        }
    };

Schema here:

https://gist.github.com/jimklo/5416348/raw/bf17eea14c35996ff1293c75a88c9224bb4e8a77/lrmi.json

This 700+k schema (!) makes very heavy use of "JSON Pointer only" JSON references. Any validation against this schema takes nearly a minute, which is too much.

The first plan is to try and expand all refs in such a schema (and check the schema syntax). In the final validator build, it can then be decided whether the full, existing mechanism is used, or a simplified one with an already expanded schema.

--- BEGIN MESSAGES --- error: err.common.additionalProperties.notAllowed level: "error" schema: {"loadingURI":"#","pointer":""} instance: {"pointer":""} domain: "validation" keyword: "additionalProperties" unwanted: ["alpha2Code","countryCode","id1"] --- END MESSAGES ---

I want to disable them . Can you suggest how ?

Hello there, you release awrong version of a class: in com.github.fge.jsonschema.core.keyword.syntax.checkers.common.EnumSyntaxChecker you call the class com.github.fge.jackson.JsonNumEquivalence that is not supposed to be in the other library since it has been changed to JsonNumEquals.
can you fix this?

The project is licensed under both the Lesser General Public License (LGPLv3) and the Apache License v2 (ASL 2.0) as per the README.md file. However, the project depends on Rhino which is released under the Mozilla Public License (MPL) only (https://github.com/mozilla/rhino). I'm no lawyer, but I don't believe that ASL is compatible with MPL; while MPL is compatible with ASL it's not reciprocal. That is ASL projects cannot contain MPL, but MPL projects can contain ASL. Can you clarify to what extent this project is released under ASL?

Headers are needed to run this jar in an OSGi environment like Eclipse.

See java-json-tools/json-schema-validator#50 for more information.

https://json-schema.org/specification-links.html#draft-7

The version of Mozilla Rhino currently being used is 1.7R4 (from 2012). The latest version is 1.7.7.1 (from 2016). If it's possible to use the newer version, please do.

As per the documentation I tried to download the jar from bintray but this is broken https://bintray.com/java-json-tools/maven/json-schema-core Can you tell me where is the official jar available for my usage?

A number of classes reference Guava classes but there is no direct dependency on Guava listed in the build.gradle. It appears that guava is being resolved transitively throught jackson coreutils.

Pattern validation is currently done with Rhino because it is availabe for Java 1.6 and fully supports ECMA 262.

But that makes JSON validation with patterns very slow. The pattern validation for one 10MB String took around 650ms on a very fast machine.

I did some further research and the same validation took 150ms using Java Pattern. But I see the ECMA point and Java Patterns are not.

But.

There is now another script engine in Java implementing ECMA 262. It's called Nashorn. Which is considerably faster. And I mean considerably. It is even faster than using Java Pattern. Validating the 10MB String took 60ms.
Ten times faster than Rhino. And nearly three times faster than Java.

But.

json-schema-core is still a Java 1.6 project and Nashorn is only available on Java 8.

But.

Why not use Rhino as fallback solution in case the Java Runtime does not offer the Nashorn Script Engine?

For the sake of performance.

https://json-schema.org/specification-links.html#draft-5

Enhancement to the core library, primarily motivated by two things:

• java-json-tools/json-schema-validator#41 -- need to expand JSON References into schemas;
• generating other schema formats from JSON Schema -- for instance Avro.

The proposed code is build around three classes:

• PointerCollector: its implementations are dependent upon the schema version being used; an implementation collects relative pointers to be used by a
• SchemaWalker: dispatches subschema collections to PointerCollector instances, and when appropriate, calls upon a
• SchemaListener: hooks into a SchemaWalker at several points; these points are: walker initialization and exit, pushd/popd, walk of current node, schema tree substitution.

Integrating this into the core has required that ALL of syntax validation be moved in there as well -- but then I wanted to do this as well --, but also all the loading mechanism -- which I wanted to do as well.

Right now:

• the SimpleSchemaWalker, which does not resolve references, works OK;
• the RecursiveSchemaWalker works only in a limited subset of cases.

Immediate goal: make RecursiveSchemaWalker handle loops. RefResolver already does, but here it is more complicated than that: the latter doesn't need to be stateful, the former needs to be.

Rhino https://mvnrepository.com/artifact/org.mozilla/rhino/1.7.14 is available and it contains a number of security patches.

See also: #27

Stacktrace:
Exception in thread "main" java.lang.ExceptionInInitializerError
at com.github.fge.jsonschema.SchemaVersion.(SchemaVersion.java:66)
at com.github.fge.jsonschema.SchemaVersion.(SchemaVersion.java:45)
Caused by: java.io.IOException: Server returned HTTP response code: 403 for URL: http://json-schema.org/draft-04/schema#
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1627)
at java.net.URL.openStream(URL.java:1041)
at com.github.fge.jackson.JsonLoader.fromURL(JsonLoader.java:117)
at com.github.fge.jsonschema.SchemaVersion.(SchemaVersion.java:64)

It seems sometimes the HTTP request to get the JSON schema will fail due to firewall settings.

If we change the code to this we can avoid this issue:

location = URI.create(uri);
URL url = new URL(uri);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.addRequestProperty("User-Agent","Test");
schema = JsonLoader.fromReader(new InputStreamReader(conn.getInputStream()));

https://json-schema.org/specification-links.html#draft-6

Original bug: java-json-tools/json-schema-validator#119

As defined by the URI javadoc:

A URI is opaque if, and only if, it is absolute and its scheme-specific part does not begin with a slash character ('/'). An opaque URI has a scheme, a scheme-specific part, and possibly a fragment; all other components are undefined.

URNs are such examples of opaque URIs. Building a URI with the existing normalization method gives a URISyntaxException.

Would be nice if validation API is extended the way that schema can be constricted directly from JsonNode, for example, consider the following swagger snippet:

responses:
  "200":
    description: Successful operation
    content:
      application/json:
        schema:
          type: array
          items:
           $ref: "#/components/schemas/animalType"
        example:
          - type: human
            id: 5ebc32178718
            sex: male
          - type: animal
            id: ba5d08e1835b
            kind: dog

After Swagger is parsed, there should be an API method to validate JsonNode of example against JsonNode of schema.

When we are validating json agains json schema, it may contain user input (e.g.) text, that we want to avoid exposing in validation exception message. Currently we have an ability to customize the error message text using com.github.fge.jsonschema.cfg.ValidationConfiguration#validationMessages, where we can exclude the actual value that failed validation.
But it has to be done for each message template separately.
Instead it would be useful to postpone message rendering, which happens in com.github.fge.jsonschema.core.report.ProcessingMessage#addArgument until it is explicitly requested, which would allow us to replace the value value in message map with masked value (e.g. replace all non-space string character with *) in application code.

From Mockito 1.9(.5) any Matcher:

This method don't do any type checks, it is only there to avoid casting in your code. This might however change (type checks could be added) in a future major release.

If the test case noFailureDoesNotTriggerEarlyExit checks with isNotNull instead of any

diff --git a/src/test/java/com/github/fge/jsonschema/core/processing/ProcessorChainTest.java b/src/test/java/com/github/fge/jsonschema/core/processing/ProcessorChainTest.java
index 072ad8b..13ea6ef 100644
--- a/src/test/java/com/github/fge/jsonschema/core/processing/ProcessorChainTest.java
+++ b/src/test/java/com/github/fge/jsonschema/core/processing/ProcessorChainTest.java
@@ -117,8 +117,8 @@ public final class ProcessorChainTest
 
         processor.process(report, input);
 
-        verify(p1).process(same(report), any(MessageProvider.class));
-        verify(p2).process(same(report), any(MessageProvider.class));
+        verify(p1).process(same(report), isNotNull(MessageProvider.class));
+        verify(p2).process(same(report), isNotNull(MessageProvider.class));
     }

it fails with:

[TestEventLogger] Gradle test > com.github.fge.jsonschema.core.processing.ProcessorChainTest.noFailureDoesNotTriggerEarlyExit FAILED
[TestEventLogger]     Argument(s) are different! Wanted:
[TestEventLogger]     processor.process(
[TestEventLogger]         same(com.github.fge.jsonschema.core.processing.ProcessorChainTest.DummyReport: success
[TestEventLogger]     ),
[TestEventLogger]         notNull()
[TestEventLogger]     );
[TestEventLogger]     -> at com.github.fge.jsonschema.core.processing.ProcessorChainTest.noFailureDoesNotTriggerEarlyExit(ProcessorChainTest.java:121)
[TestEventLogger]     Actual invocation has different arguments:
[TestEventLogger]     processor.process(
[TestEventLogger]         com.github.fge.jsonschema.core.processing.ProcessorChainTest.DummyReport: success
[TestEventLogger]     ,
[TestEventLogger]         null
[TestEventLogger]     );
[TestEventLogger]     -> at com.github.fge.jsonschema.core.processing.ProcessorChain$ProcessorMerger.process(ProcessorChain.java:190)
[TestEventLogger]         at com.github.fge.jsonschema.core.processing.ProcessorChainTest.noFailureDoesNotTriggerEarlyExit(ProcessorChainTest.java:121)
[TestEventLogger]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

The test setup needs a return value for p1 if process is executed such as:

diff --git a/src/test/java/com/github/fge/jsonschema/core/processing/ProcessorChainTest.java b/src/test/java/com/github/fge/jsonschema/core/processing/ProcessorChainTest.java
index 072ad8b..d9764be 100644
--- a/src/test/java/com/github/fge/jsonschema/core/processing/ProcessorChainTest.java
+++ b/src/test/java/com/github/fge/jsonschema/core/processing/ProcessorChainTest.java
@@ -114,11 +114,13 @@ public final class ProcessorChainTest
 
         final MessageProvider input = mock(MessageProvider.class);
         final ProcessingReport report = new DummyReport(LogLevel.DEBUG);
+        
+        when(p1.process(report, input)).thenReturn(input);
 
         processor.process(report, input);
 
-        verify(p1).process(same(report), any(MessageProvider.class));
-        verify(p2).process(same(report), any(MessageProvider.class));
+        verify(p1).process(same(report), isNotNull(MessageProvider.class));
+        verify(p2).process(same(report), isNotNull(MessageProvider.class));
     }
 
     private static final class DummyReport

With this setup the test case passed.