eclipse-vertx / vertx-json-schema Goto Github PK

Version

4.3.3

Context

https://vertx.io/docs/vertx-json-schema/java/#_parse_a_schema describes how to use

JsonSchema schema = JsonSchema.of(object);

However, the interface is package-private and can not be used by other code.

Do you have a reproducer?

Using

 <dependency>
            <groupId>io.vertx</groupId>
            <artifactId>vertx-json-schema</artifactId>
            <version>4.3.3</version>
        </dependency>

Steps to reproduce

New maven project with dependency, write line:

JsonSchema schema = JsonSchema.of(null);

Extra

The current readme is insufficient on providing enough context on how to use this lib

Needs more documentation and examples

• vertx-dependencies vert-x3/vertx-dependencies#37
• vertx-stack (depchain + docs) vert-x3/vertx-stack#99
• Website doc link vert-x3/vertx-web-site#335
• Langs PR?

The meta schema 2020-12 does make use of this functionality, which render the meta validation useless for 2020-12.

We need to implement this functionality and pass the disabled TCK tests:

https://github.com/eclipse-vertx/vertx-json-schema/blob/master/src/test/resources/unsupported-tck-tests.properties#L99-L108

Version

4.3.4

Context

The analysis tool “SpotBugs 4.7.3” pointed the following information out for 35 source code places.

💭 NullPointerException caught

It is occasionally useful to determine if passed input data are valid at all according to selected schemas.
💭 But I would like to achieve a higher level data processing service.

Some data processing efforts were performed until the final validation result will be returned.
🤔 Thus I hope that parsing efforts do not need to be repeated for more desirable data reuse.
I would prefer to work with a data model which would support further processing in more convenient ways than the programming interface “Vert.x JsonObject”.
🔮 How will the chances evolve to add corresponding functions and classes?

Version

    <dependency>
        <groupId>io.vertx</groupId>
        <artifactId>vertx-json-schema</artifactId>
        <version>4.0.0</version>
    </dependency>

Context

I'm attempting to validate an asyncApi json object, against the asyncApi schema using schema.validateAsync (validateSync fails too)

The AsyncAPI schema is here https://raw.githubusercontent.com/asyncapi/asyncapi/master/versions/2.0.0/schema.json

When validateAsync gets to checking the propertyNames in my json object, I get the following exception:

input don't match any of types [STRING]
ValidationException{message='input don't match any of types [STRING]', keyword='type', input={"topic":{"subscribe":{"message":{"payload":{"type":"string"}}}}}, schema=io.vertx.json.schema.common.SchemaImpl@5bdad412, scope=urn:vertxschemas:05718eb8-d4ff-4d36-92ce-7649ce0e5014#}
	at io.vertx.json.schema.ValidationException.createException(ValidationException.java:54)
	at io.vertx.json.schema.draft7.TypeValidatorFactory$TypeValidator.validateSync(TypeValidatorFactory.java:92)
	at io.vertx.json.schema.common.SchemaImpl.runSyncValidator(SchemaImpl.java:200)

Do you have a reproducer?

I've shrunk down the schema and json to a small recreate:

specDefinition:

{
  "title": "AsyncAPI 2.0.0 schema.",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "object",
  "properties": {
    "channels": {
      "type": "object",
      "propertyNames": {
        "type": "string"  -> swap this to be object and it works fine in vertx, but fails for other validators
      }
    }
  }
}

jsonData:

{
  "channels": {
    "topic": {
      "subscribe": {
          "message": {
            "payload": {
              "type": "string"
            }
          }
        }
      }
  }
}

The above object and spec validates OK in eg https://www.liquid-technologies.com/online-json-schema-validator or using a different Java json schema validator.

Steps to reproduce

SchemaRouter schemaRouter = SchemaRouter.create(vertx, new SchemaRouterOptions());
SchemaParser schemaParser = Draft7SchemaParser.create(schemaRouter);

//specDefinition + jsonData - Files read in using vertx.fileSystem().readFile

Schema schema = schemaParser.parse(specDefinition.toJsonObject());
schema.validateAsync(jsonData.toJsonObject()).onFailure(error -> {
error.printStackTrace();
});

Extra

When the validator iterates through the valid SchemaTypes here

I think that only a String is a valid type but the json being checked is of type object - {"topic":{"subscribe":{"message":{"payload":{"type":"string"}}}}}

propertyNames should validate the fields within the json object, not the whole object.

👀 I took another look at the implementation of the method “validate”.
Thus I noticed that source code like “if (schema.containsKey("…"))” is used at some places.
The passed key word is always checked.

🤔 I imagine that unnecessary condition checks can be avoided if an other algorithm can be applied.
Thus I propose to consider data processing alternatives by working with Petri nets.
🔮 I am curious on further software evolution in such a design direction.

Get the following error,

Provided value don't match pattern, Input doesn't match one of allowed values of enum: [Ljava.lang.Object;@470aedb8]. Our enum in the OpenAPI spec is enum: [file].

Version

4.0.0.CR1

Steps to reproduce

Pass an incorrect enum value

Version

4.3.1, 4.4.1, etc.

Context

I've run into a problem with SchemaRepository that seems to stem from concurrent access to the repository. Having a look at the code I think it could be fixed by changing the lookup map to a ConcurrentHashMap rather than a simple HashMap.

My production case is: I have many different schemas stored in a database. The application is a message-processor; it receives a message, reads the schema from the database (in text format) and validates them using the Validator. Often, near application startup we see exceptions that don't make much sense, such as a call to dereference followed immediately by a call to validator with that same key throws Exception in thread "Main" java.lang.IllegalArgumentException: Unknown $ref: propName_2

Do you have a reproducer?

https://github.com/akleiman/vertx-jsonvalidator-concurrency-reproducer
run with gradle run, it was built on Java 20. the first lines of the output are what I see in my production code. In production I'm not using virtual threads in my production code, I used it in the reproducer for simplicity and so I could spawn many different threads at once.

Reproducers are very helpful for contributors and will likely help them fixing your bug faster.

Steps to reproduce

1. Set up a SchemaRepository to be used concurrently
2. On multiple threads/executors:
3. Load a schema (only tested by loading from a string)
   4.Dereference the schema
4. Obtain the validator
5. Some of the calls to validator will fail, as if it didn't know the URL that was just registered in dereference.

Extra

Java 17 and 20.

Version

4.4.1

Context

After upgrading from version 4.2.7 to 4.4.1, I have noticed that validating a number field which has 'multipleOf 0.001' defined in Open API 3 fails for numbers like 1.001, 1.01, 1.02, but passes for numbers like 0.001, 0.002.

Do you have a reproducer?

No, but the changes made in the code are transparent.

Steps to reproduce

The bug was already reported in this issue:
vert-x3/vertx-web#2105

But it was reverted in this commit:
a51e3e3

Question

Since the OpenAPI3SchemaParser class which includes the MultipleOfValidatorFactory is marked as deprecated, is there an alternative solution planned for future releases?

In order to support the latest openAPI (the migration guide) mentions 2020-02

https://www.openapis.org/blog/2021/02/16/migrating-from-openapi-3-0-to-3-1-0

Look at vert-x3/vertx-web#1816 for more details

Version

4.3.5

Context

The validation using pattern FASTDATETIME fails to catch invalid dates where the day is within the 1-31 range, but the month does not have 31 days (e.g. 31 November).

Solution

The definition of private static boolean testDateTime(String value) here
should be rewritten to return

Instant.parse(value);

instead of

FASTDATETIME.matcher(value).find();

Version

4.3.7

Context

I assume that this line should actually be

    Objects.requireNonNull(this.draft, "'draft' cannot be null either #schema.$draft or options.draft");

instead of

    Objects.requireNonNull(schema, "'draft' cannot be null either #schema.$draft or options.draft");

Otherwise the error message makes no much sense and the non-nullness of schema was already tested on L25

Questions

Version

4.1.x and 4.3.8 it is the same behaviour

Context

I encountered an exception while using the yourkit profiler to see CPU hot spots.
The filling in of the stack traces is taking too much CPU. Expect we can suppress that or try to avoid the throwing of exceptions and instead return false. or else suppress the filling in of the large stack trace in the exception.

Do you have a reproducer?

Using a json schema to validateSync, with a oneOf and many required options.

there is an example in this json schema:
{"type":"object","properties":{"schema":{"type":"object","properties":{"A":{"type":"object","properties":{"ttl":{"type":"integer"},"ipV4":{"type":"string"}},"required":["ttl","ipV4"]},"AAAA":{"type":"object","properties":{"ttl":{"type":"integer"},"ipV6":{"type":"string"}}},"CNAME":{"type":"object","properties":{"ttl":{"type":"integer"},"dn":{"type":"string"}},"required":["ttl","dn"]},"NS":{"type":"object","properties":{"ttl":{"type":"integer"},"dn":{"type":"string"}},"required":["ttl","dn"]},"PTR":{"type":"object","properties":{"ttl":{"type":"integer"},"dn":{"type":"string"}},"required":["ttl","dn"]},"HINFO":{"type":"object","properties":{"ttl":{"type":"integer"},"cpu":{"type":"string"},"os":{"type":"string"}},"required":["ttl","cpu","os"]},"MX":{"type":"object","properties":{"ttl":{"type":"integer"},"preference":{"type":"integer"},"exchange":{"type":"string"}},"required":["ttl","preference","exchange"]},"NAPTR":{"type":"object","properties":{"ttl":{"type":"integer"},"order":{"type":"integer"},"preference":{"type":"integer"},"flags":{"type":"string"},"service":{"type":"string"},"regexp":{"type":"string"},"replacement":{"type":"string"}},"required":["ttl","order","preference"]},"SRV":{"type":"object","properties":{"ttl":{"type":"integer"},"priority":{"type":"integer"},"weight":{"type":"integer"},"port":{"type":"integer"},"target":{"type":"string"}},"required":["ttl","priority","weight","port","target"]},"TXT":{"type":"object","properties":{"ttl":{"type":"integer"},"txt":{"type":"string"}},"required":["ttl","txt"]}},"oneOf":[{"required":["A"]},{"required":["AAAA"]},{"required":["CNAME"]},{"required":["NS"]},{"required":["PTR"]},{"required":["HINFO"]},{"required":["MX"]},{"required":["NAPTR"]},{"required":["SRV"]},{"required":["TXT"]}]},"policy":{"type":"object","properties":{"priority":{"type":"integer"},"weight":{"type":"integer"},"effective-start":{"type":"string","format":"date-time"},"effective-end":{"type":"string","format":"date-time"},"tod-start":{"type":"string","format":"time"},"tod-end":{"type":"string","format":"time"},"dow":{"type":"object","properties":{"sun":{"type":"boolean","default":false},"mon":{"type":"boolean","default":false},"tue":{"type":"boolean","default":false},"wed":{"type":"boolean","default":false},"thu":{"type":"boolean","default":false},"fri":{"type":"boolean","default":false},"sat":{"type":"boolean","default":false}}},"timezone":{"type":"string"}}}},"required":["schema"]}

Reproducers are very helpful for contributors and will likely help them fixing your bug faster.

• Link to github project/gist

Steps to reproduce

Run the schema validateSync function with some like above example, and step thru debugger with each validationexception getting thrown.

Extra

• Anything that can be relevant such as OS version, JVM version

Questions

I took another look at the capabilities of the following software.

Version

4.3.4

Context

I constructed the following JSON schema file.

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Calculation information",
  "description": "settings for calculations",
  "$id": "https://example.com/schemas/calculation_preferences",
  "type": "object",
  "required": ["context"],
  "properties":
  {
    "context":
    {
      "type": "object",
      "oneOf":
      [
        {
          "allOf":
          [
            {
              "properties":
              {
                "label": {"const": "GTIN"}
              }
            },
            {
              "type": "array",
              "minItems": 1,
              "items":
              {
                "type": "string",
                "pattern": "^[0-9]{13}$"
              }
            }
          ]
        },
        {
          "allOf":
          [
            {
              "properties":
              {
                "label": {"const": "selection group"}
              }
            },
            {
              "type": "array",
              "minItems": 1,
              "items":
              {
                "type": "string",
                "minLength": 1
              }
            }
          ]
        }
      ]
    }
  }
}

Test data example:

{
  "context":
  {
    "GTIN": [ "1234567890123" ]
  }
}

💭 Now I am wondering about the following error information.

Markus_Elfring@…:…/Schema-Parsing3> java -jar target/Schema-Parsing3-0.0.1-SNAPSHOT.jar ../Context-test_fragment1.json
Validation errors
error|keyword|keywordLocation|instanceLocation
Property "context" does not match schema|properties|#/properties|#
Instance does not match exactly one subschema (0 matches)|oneOf|#/properties/context/oneOf|#/context
Instance does not match every subschema|allOf|#/properties/context/oneOf/0/allOf|#/context
Instance type object is invalid. Expected array|type|#/properties/context/oneOf/0/allOf/1/type|#/context
Instance does not match every subschema|allOf|#/properties/context/oneOf/1/allOf|#/context
Instance type object is invalid. Expected array|type|#/properties/context/oneOf/1/allOf/1/type|#/context
java.lang.RuntimeException: JSON schema validation failure
        at main.java.….app.JsonSchemaCheck.reportErrors(CLI.java:47)
        at main.java.….app.JsonSchemaCheck.test4(CLI.java:116)
        at main.java.….app.JsonSchemaCheck.call(CLI.java:129)
        at main.java.….app.JsonSchemaCheck.call(CLI.java:20)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2314)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2316)
        at picocli.CommandLine.execute(CommandLine.java:2078)
        at main.java.….app.JsonSchemaCheck.main(CLI.java:134)

🤔 I would appreciate further advices and solution ideas for these reported data type mismatches.

Version

Which version(s) did you encounter this bug ?
4.2.5

Context

The exception message has only this

[Bad Request] Validation error for body application/json: No schema matches

while other schema errors tell you what is missing, or what keyword is not being followed.

Version

4.3.5

Context

The API documentation provides the following information.

• vertx-json-schema/src/main/java/io/vertx/json/schema/Validator.java

  Line 37 in eba3d06

  * Validate a given input against the initial schema

• vertx-json-schema/src/main/java/io/vertx/json/schema/Validator.java

  Line 19 in eba3d06

  * A validator, validates some input object using a well known schema.

💭 Thus I find an other wording more appropriate than “To validate a schema:” in the user documentation.

Version

4.0.2

Context

Encountered a validation exception when validating a hostname that had a domain which started with a numeric value. After checking against other implementations of json-schema validation, this seems to be unique to VertX's implementation of it.

RFC requirements originally specified that hostnames could not start with numeric values, but this has been updated as of RFC 1123.

Steps to reproduce

1. create a schema with the following properties:
   { "type": "object", "properties": { "host": { "type": "string", "format": "hostname" } }
2. Validate the following payload against it:
   { "host":"www.3gppnetwork.org" }

Questions

N/A

Version

4.1.5

Context

I encountered an exception which looks suspicious while I use Parent schema for Object which Properties are reference to child Schema.

_

"type": "object",
"properties": {
"prop1": {
"$ref": "schema-child.json#/properties/prop1"
},
"prop2": {
"$ref": "schema-child.json#/properties/prop2"
}

_

with Child schema:

"$schema": "https://json-schema.org/draft/2019-09/schema",
"type": "object",
"properties": {
"prop1": {
"description": "prop 1",
"type": "string",
"format": "uuid"
},
"prop2": {
"description": "prop 2",
"type": "string"
}
}

The External references are resolved by resolveExternalRef(final JsonPointer pointer, final JsonPointer scope, final SchemaParser schemaParser) function of SchemaRouterImpl.

The first properties ("prop1") schema resolution works fine.
However, the resolved schema is added to externalSchemasSolving Map with key = schema-child.json

Then, at the 2nd property resolution ("prop2"), the externalSchemasSolving Map is used to get schema but using same key : URI of the Pointer Without Fragment ==> schema-child.json
So, the resolved schema of the 1st schema is returned instead the correct schema.

The hereunder JSON validation FAILED because "prop2" value is NOT UUID. (that is only applicable to "prop1")

{
"prop1" : "123e4567-e89b-42d3-a456-556642440000",
"prop2" : "prop2Value"
}

Error:
"ValidationException{message='Provided value don't match pattern', keyword='pattern', input=prop2Value, schema=io.vertx.json.schema.common.SchemaImpl@521aa3d3, inputScope=#/prop2}"

I guess the full pointer should be used instead of the URIWithoutFragment as externalSchemasSolving Map Key.

Do you have a reproducer?

Not a Git project , but samples Json schema + test class (Attached)

Steps to reproduce

Cf test class attached to this Issue

Extra

Attached Sample Json schema Files (to be renamed to .json)

• Test class (IssueDemonstratorTest)

schema-child.txt
schema-parent.txt
IssueDemonstratorTest.txt

This issue is intended to collect thoughts about an eventual discriminator support. https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.3.md#schemaObject

The OutputUnit has a method "getErrors()" which should return an empty list if no errors are present instead of null.

Hi,

I've recently needed to compare the performance and functionality of this and other JVM based JSON validation libraries, and I thought you might like to see how this implementation compares. I've published to the code and the results here: https://github.com/creek-service/json-schema-validation-comparison.

While I've spent time trying to get the best out of each library under test, it's possible that the results for this implementation could be improved if someone with more in-depth knowledge were to look into it. If you feel I'm not showing in its best light, please feel free to raise PRs to fix issues and improve your score.

Please feel free to link to the code and results.

I hope this is of some use to you.

Thanks,

Andy

Currently, the library only handles OBJECT/ARRAY if the input object has type JsonObject/JsonArray.

This behavior causes confusion on es4x since the @VertxGen API to validate accepts Object in. Since the java host accepts any type, graaljs will use the lowest conversion possible type: JS Object -> j.u.Map and validations will always fail as Map isn't allowed.

I think the "fix" should be to allow the usage of Map and List where OBJECT and ARRAY are expected. This has 2 benefits imho:

1. it will address the es4x issue automatically
2. it allows the usage of the library outside the vert.x ecosystem where json isn't defined as the vert.x specific types but map and list are quite common.

Version

v4.3.0

Context

I've tried to resolve the references of the OPEN API provided by Swagge here : https://editor.swagger.io/

JsonObject jo = yamlFileToJO(filePath); JsonSchema jsonSchema = JsonSchema.of(jo); JsonObject joResolved = jsonSchema.resolve();

The resolution failed with follwoing error:
"Can't resolve '{"type":"integer","format":"int64"}#/definitions/Category', only internal refs are supported."

Issue seems located to line 251 of https://github.com/eclipse-vertx/vertx-json-schema/blob/master/src/main/java/io/vertx/json/schema/impl/JsonObjectSchema.java

because the 'Category' Schema contains a property named "id".

Do you have a reproducer?

Cf code above

Steps to reproduce

Cf code above

Extra

Currently, we skip about 300 tests from the TCK:

https://github.com/eclipse-vertx/vertx-json-schema/blob/master/src/test/resources/unsupported-tck-tests.properties

This is a general purpose issue to track pull request to address the issues. The issues should be considered "good first issue" kind of tasks.

To address an unsupported test, first one needs to understand the test, for example:

draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 29\ days\ in\ February\ (normal)=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 30\ days\ in\ February\ (leap)=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 31\ days\ in\ April=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 31\ days\ in\ June=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 31\ days\ in\ November=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 31\ days\ in\ September=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 32\ days\ in\ August=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 32\ days\ in\ December=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 32\ days\ in\ January=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 32\ days\ in\ July=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 32\ days\ in\ March=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 32\ days\ in\ May=skip
draft2019-09/optional/format/date/validation\ of\ date\ strings/a\ invalid\ date\ string\ with\ 32\ days\ in\ October=skip

shows that the code we have here:

Is actually not fully correct. It does validate that the input follows the right notation but doesn't validate edge cases like mentioned in the case above.

To fix this, that method must be refactored to perform the complete validation, not just the format string.

While this issue can be solved without much thinking, extra attention should be used to verify the performance/thread safety of the implementation as the code can be on the "hot" path of the execution of a vert.x application.

We will be availabe to support new contributors to work on these issues!

Describe the feature

We need a way to trace at which point of the input we are when the validation is executed. This is necessary to include in the ValidationException the JsonPointer of the input where the validation failed.

For more info vert-x3/vertx-web#1843

Version

4.1.2

Context

ConcurrentModificationExceptions occur often with many schemas loaded. This is probably the same ConcurrentModificationException seen in the earlier issue.

Note that removing most schemas except ChfInfoList (which had the NullPointerExceptions in the earlier ticket, but does not reproduce here) and LmfInfo (which has most of the ConcurrentModificationExceptions) and the schemas they depend on will fix this issue.

Do you have a reproducer?

https://github.com/scardwell15/vertx-schema-tester

Reproducers are very helpful for contributors and will likely help them fixing your bug faster.

• Link to github project/gist

Steps to reproduce

1. Load many schemas in tests and try to assert validations.
2. Notice concurrent modification exceptions in vertx-json-schema, mostly in LmfInfoNGTest.

Extra

java 11 on feren OS

This blocks, downstream projects to generate proper polyglot bindings, e.g.:

reactiverse/es4x#564

Some data can be stored also in schema repositories.
👀 I am looking for ways to retrieve selected schema information from this storage system.
🔮 Which programming interfaces will support such functionality?

Version

4.0-SNAPSHOT

Context

Currently pom.xml define a reference to slf4j-simple in default scope rather that test scope.

Moved from vert-x3/vertx-web#1690 at @slinkydeveloper's request,

Version

4.0.0-milestone4

Do you have a reproducer?

Using the below schema, if I make the below request with an invalid field name such that if fails to match any of the anyOf types, I get the error, $.data.relationships.source.data.id: object found, string expected. However, the field can be either a string or an object.

relationships:
  type: object
  required: [source]
  properties:
    source:
      anyOf:
        - type: object
          required: [data]
          properties:
            data:
              type: object
              required: [id]
              properties:
                id:
                  type: string
        - type: object
          required: [data]
          properties:
            data:
              type: object
              required: [id]
              properties:
                id:
                  anyOf:
                    - type: string
                    - type: object
                      required: [pId, rId]
                      properties:
                        pId:
                          type: string
                        rId:
                          type: string

"relationships": {
  "source": {
    "data": {
      "type": "animal",
      "id": {
        "xxx": "x",
        "rId": "y"
      }
    }
  }
}

Describe the feature

We already have resolveRef for querying schemas, but as in the title implies, the SchemaRouter needs a asynchronous way to addSchemas to it.

Use cases

I have an application that gets gets schemas from various places. These schemas are versioned, so they can be safely cached. I created a SchemaRouter that persists the schemas to the disk. Problem is, in my implementation, addSchema can fail. My current workaround is wrapping addSchema calls unto a executeBlocking Promise.

Contribution

I can contribute a small patch if needed. I don't know the full implication of such changes, but since this package is marked as "Technical Preview", such change should be expected by users.

The current package name is io.vertx.ext.json.schema, we are actually avoiding to use ext whenever possible in new project. So we should change the project package before it is released.

The programming interface “Vert.x SchemaRepository” supports a few validation functions.
💭 I would like to be able to identify the desired start validation schema by the standard property “$id”.
🔮 How will the chances evolve to add corresponding functions?

Questions

I took another look at the capabilities of the following software.

Version

4.3.3

Context

Now I wonder about the error message “The import io cannot be resolved” (for a while).
The required software components are also not displayed in the folder “Project and External Dependencies”.

How should the dependency resolution be fixed here? 🤔

Steps to reproduce

1. I constructed another Gradle project example by the means of the provided wizard.
2. I added the specification “compile 'io.vertx:vertx-json-schema:4.3.3'” to the block “dependencies” in the file “build.gradle” as this configuration is mentioned in the documentation.
3. A demonstration source file contains the following code.

   package examples;

   import io.vertx.core.json.JsonObject;
   import io.vertx.json.schema.*;

   public class JsonSchemaExamples {
   …
   }

Thus I copied the shown declarations to my own source file.

Extra

Eclipse IDE for Java Developers 2022-06 (4.24.0)
Eclipse Plug-ins for Gradle 3.1.6.v20220511-1359

Questions

Refer to io.vertx.json.schema.common.SchemaImpl which has a list of validators iterated during processing.

https://github.com/eclipse-vertx/vertx-json-schema/blob/master/src/main/java/io/vertx/json/schema/common/SchemaImpl.java#L171

https://github.com/eclipse-vertx/vertx-json-schema/blob/master/src/main/java/io/vertx/json/schema/common/BaseFormatValidatorFactory.java#L103

Am adding a custom pattern on a date-time property to override the default DATETIME pattern regex, however, when processing, notice that the default BaseFormatValidatorFactory FormatValidator still remains in the validator list along with the custom PatternValidatorFactory PatternValidator that was added.

Is it a bug? Why is the default format validator there, as expected it to be replaced / removed by the custom one.

Version

4.1.2

Context

I encountered an exception which looks suspicious while testing validateSync with a schema.

Do you have a reproducer?

Yes there is a testng but it is in private project, not portable as a public project. But the existing vertx unit test might be enhanced to use a custom date time pattern to see that the default formatter is not removed from the validator list.

Steps to reproduce

Extra

The schema am using is this one:

                "effective-start": {
                    "type": "string",
                    "format": "date-time",
                    "pattern" : "^\\d{4}\\-\\d{2}\\-\\d{2}[\\s]\\d{2}:\\d{2}:\\d{2}"
                }

`

Questions

The JsonObjectSchema.resolve() only resolves References inside Jsonbject.

However, JsonArray may also contains reference(s).

Like for example, "Parameters" Field in OpenAPI: https://spec.openapis.org/oas/latest.html#operation-object

"parameters":[{"$ref":"#/components/parameters-query/aQeueryKey"}]

Version

v 4.3.0

Context

Do you have a reproducer?

No

Steps to reproduce

1. Create JsonSchema instance from JsonObject containning arrays of Refs
2. Call jsonSchema.resolve();
3. Refs inside Arrays are not resolved

Extra

Version

Used following Dependency of vertx in pom.xml

<dependency>
     <groupId>io.vertx</groupId>
     <artifactId>vertx-json-schema</artifactId>
     <version>4.2.0.CR1</version>
</dependency>

<dependency>
    <groupId>io.vertx</groupId>
    <artifactId>vertx-core</artifactId>
    <version>4.2.0.CR1</version>
</dependency>

Context

While running the Vertx for doing json validation using following code, we noticed that after the execution, there are some Vertx threads which are not closed properly, so even if function returns back to the main method, main method is not able to terminate the java application properly

Do you have a reproducer?

Isolated this problem using the following java code

import java.io.File;
import java.util.concurrent.CompletableFuture;
import io.vertx.core.Vertx;
import io.vertx.core.json.pointer.JsonPointer;
import io.vertx.json.schema.Schema;
import io.vertx.json.schema.SchemaParser;
import io.vertx.json.schema.SchemaRouter;
import io.vertx.json.schema.SchemaRouterOptions;
import io.vertx.core.json.JsonObject;
import org.apache.commons.lang3.ThreadUtils;

public class JSONValidatorVertx {

    /**
     * This method is used for json validation
     * 
     * @param Data: json data on which validations needs to be performed
     * @param jsonSchema: file path to json schema 
     * @return Boolean
     * @throws Exception: If any error occurs during json validation
     */
    private static Boolean validateJSON(String data, String jsonSchema) throws Exception {
        long beforeTimestamp = System.currentTimeMillis();
        try {
          SchemaRouter ROUTER = SchemaRouter.create(Vertx.vertx(), new SchemaRouterOptions());
          SchemaParser PARSER = SchemaParser.createDraft201909SchemaParser(ROUTER);
          String jsonSchemaContent =  AppUtility.readFileContent(jsonSchema);
          JsonPointer pointer = JsonPointer.fromURI(new File(jsonSchema).toURI());
          Schema schema = PARSER.parse(new JsonObject(jsonSchemaContent), pointer);
          CompletableFuture<Void> future = new CompletableFuture<>();
          schema.validateAsync(new JsonObject(data)).onSuccess(future::complete).onFailure(future::completeExceptionally);
          future.get();
          System.out.println("End of try statment");
      } catch (Exception headerException) {
           System.out.println(headerException.toString());
           throw headerException;
       }
        System.out.println("Performed json validation in " + (System.currentTimeMillis() - beforeTimestamp) + " ms");
        return true;
    }

    public static void main( String[] args ) throws Exception {
        System.out.println( "Hello! You have triggered JSON validator");
        try {
            // Use function to read input json data 
            String inputData = AppUtility.readFileContent(args[0]);
            //String inputValidationSchema = AppUtility.readFileContent(args[1]);
            // Call validation function
            Boolean result = validateJSON(inputData, args[1]);
            String content = new String("JSON validation Result: ").concat(result.toString());
            for (Thread t : ThreadUtils.getAllThreads()) {
                System.out.println(t.getName() + ", " + t.isDaemon());
          }
            System.out.println(content);
        } catch (Exception ex){
            System.out.println("Exception occurred: \n " + ex.getStackTrace());
            throw ex;
        }
    }

}

Output after running gives us following:

Hello! You have triggered JSON validator
End of try statment
Performed json validation in 1010 ms
Reference Handler, true
Finalizer, true
Signal Dispatcher, true
Attach Listener, true
Notification Thread, true
main, false
vertx-blocked-thread-checker, true
Thread-2, true
vert.x-eventloop-thread-0, false
vert.x-internal-blocking-0, false
vert.x-internal-blocking-1, false
Common-Cleaner, true
JSON validation Result: true

Even though the main get the control back from the validateJSON, java application does not terminate gracefully and had to be terminated forcefully by the user

JVM version

openjdk 15.0.2 2021-01-19
OpenJDK Runtime Environment AdoptOpenJDK (build 15.0.2+7)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 15.0.2+7, mixed mode, sharing)

Version

4.3.4

Context

I noticed that a code fragment like “…Location + "/" + …” is used within loops at several places.
💭 I suggest to perform such appending of slashes with the help of additional string variables before looping.

JsonRef.resolve(...) and SchemaRepository.resolve(...) (which internally calls JsonRef) are used to create a dereferenced representation of a passed JSON schema. This means that properties like "$ref" are replaced by their actual value.

JSON Schema allows to have circular references, which makes it possible to build a schema with an infinite depth. If we are calling toString() on these JsonObjects with circular references, we ran obviously into a StackOverflowError.

In my opinion we have these options: resolve(...) methods ...

1. ... don't longer return a JsonObject. Maybe a ResolvedSchema object (better name is welcome), which don't offer any kind of serialization methods.
2. ... return still a JsonObject, but all serialization methods are overridden and throw an UnsupportedOperationException.

https://github.com/eclipse-vertx/vertx-json-schema/runs/1822168214

This software supports parsing, validation and further data processing also for JSON schemas to some degree.
👀 I am looking for a program variant which will combine available APIs into a convenient command line interface.
🔮 How will corresponding development possibilities evolve?

Version

4.1.1

Context

While updating a project to use vertx-json-schema to validate JSON schemas based on TS 3GPP documents read from files, NullPointerExceptions and ConcurrentModificationExceptions are encountered. These exceptions are inconsistent, and always happen in Vertx code. Additionally, sometimes, the "trying to execute sync validation in async state" exception is encountered when calling validateAsync.

Do you have a reproducer?

No

Steps to reproduce

1. Write schemas that use $refs. The project has many, many schemas that reference other schemas.
2. Use validateAsync to validate many of the schemas. The project contains test classes for 20-30 of them, each test class containing test methods that assert that validation works as expected.

Extra

validation.JsonValidationException: ValidationException{message='additionalProperties schema should match', keyword='additionalProperties', input={"additionalProperties":{"supiRangeList":[{"start":"111","end":"999","pattern":"pattern"}],"gpsiRangeList":[{"start":"123","end":"789","pattern":"pattern-value"}],"plmnRangeList":[{"start":"11122","end":"11133","pattern":"pattern-value"}],"groupId":"123456789","primaryChfInstance":"6f134298-939a-47d6-9566-fd0030517ac2","secondaryChfInstance":"66ac6eea-db7c-44e4-a8e8-2a5d6e5184ef"}}, schema=io.vertx.json.schema.common.SchemaImpl@7e7a6f81, inputScope=#} at validation.JsonValidationUtils.validateObject(JsonValidationUtils.java:74) at validation.ts3gpp.extensions.ChfInfoListValidator.validate(ChfInfoListValidator.java:41) at validation.ts3gpp.extensions.ChfInfoListNGTest.validate(ChfInfoListNGTest.java:53) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:124) at org.testng.internal.Invoker.invokeMethod(Invoker.java:583) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:719) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:989) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.privateRun(TestRunner.java:648) at org.testng.TestRunner.run(TestRunner.java:505) at org.testng.SuiteRunner.runTest(SuiteRunner.java:455) at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:450) at org.testng.SuiteRunner.privateRun(SuiteRunner.java:415) at org.testng.SuiteRunner.run(SuiteRunner.java:364) at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52) at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:84) at org.testng.TestNG.runSuitesSequentially(TestNG.java:1208) at org.testng.TestNG.runSuitesLocally(TestNG.java:1137) at org.testng.TestNG.runSuites(TestNG.java:1049) at org.testng.TestNG.run(TestNG.java:1017) at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:135) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:193) at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:94) at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:146) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345) at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418) Caused by: ValidationException{message='additionalProperties schema should match', keyword='additionalProperties', input={"additionalProperties":{"supiRangeList":[{"start":"111","end":"999","pattern":"pattern"}],"gpsiRangeList":[{"start":"123","end":"789","pattern":"pattern-value"}],"plmnRangeList":[{"start":"11122","end":"11133","pattern":"pattern-value"}],"groupId":"123456789","primaryChfInstance":"6f134298-939a-47d6-9566-fd0030517ac2","secondaryChfInstance":"66ac6eea-db7c-44e4-a8e8-2a5d6e5184ef"}}, schema=io.vertx.json.schema.common.SchemaImpl@7e7a6f81, inputScope=#} at io.vertx.json.schema.ValidationException.create(ValidationException.java:71) at io.vertx.json.schema.common.PropertiesValidatorFactory.fillAdditionalPropertyException(PropertiesValidatorFactory.java:111) at io.vertx.json.schema.common.PropertiesValidatorFactory.access$200(PropertiesValidatorFactory.java:32) at io.vertx.json.schema.common.PropertiesValidatorFactory$PropertiesValidator.lambda$validateAsync$0(PropertiesValidatorFactory.java:206) at io.vertx.core.impl.future.ComposeTransformation.onFailure(ComposeTransformation.java:50) at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:78) at io.vertx.core.impl.future.FutureImpl.addListener(FutureImpl.java:160) at io.vertx.core.impl.future.FutureBase.compose(FutureBase.java:104) at io.vertx.core.Future.recover(Future.java:210) at io.vertx.json.schema.common.PropertiesValidatorFactory$PropertiesValidator.validateAsync(PropertiesValidatorFactory.java:206) at io.vertx.json.schema.common.SchemaImpl.runAsyncValidators(SchemaImpl.java:173) at io.vertx.json.schema.common.SchemaImpl.validateAsync(SchemaImpl.java:121) at io.vertx.json.schema.common.SchemaImpl.validateAsync(SchemaImpl.java:48) atvalidation.JsonValidationUtils.validateObject(JsonValidationUtils.java:60) ... 32 more Caused by: java.lang.NullPointerException at io.vertx.json.schema.common.SchemaImpl.runAsyncValidators(SchemaImpl.java:171) at io.vertx.json.schema.common.SchemaImpl.validateAsync(SchemaImpl.java:121) at io.vertx.json.schema.common.RefSchema.lambda$validateAsync$1(RefSchema.java:79) at io.vertx.core.impl.future.ComposeTransformation.onSuccess(ComposeTransformation.java:38) at io.vertx.core.impl.future.FutureBase.emitSuccess(FutureBase.java:61) at io.vertx.core.impl.future.SucceededFuture.addListener(SucceededFuture.java:82) at io.vertx.core.impl.future.FutureBase.compose(FutureBase.java:104) at io.vertx.core.impl.future.SucceededFuture.compose(SucceededFuture.java:27) at io.vertx.json.schema.common.RefSchema.validateAsync(RefSchema.java:65) at io.vertx.json.schema.common.PropertiesValidatorFactory$PropertiesValidator.validateAsync(PropertiesValidatorFactory.java:205)

schema below

Caused by: io.vertx.json.schema.NoSyncValidationException: Trying to execute validateSync() for a Validator in asynchronous state at io.vertx.json.schema.common.BaseMutableStateValidator.checkSync(BaseMutableStateValidator.java:59) at io.vertx.json.schema.common.SchemaImpl.validateSync(SchemaImpl.java:126) at io.vertx.json.schema.common.RefSchema.validateSync(RefSchema.java:104) at io.vertx.json.schema.common.ItemsValidatorFactory$ItemsValidator.validateSync(ItemsValidatorFactory.java:49) at io.vertx.json.schema.common.SchemaImpl.runSyncValidator(SchemaImpl.java:193) at io.vertx.json.schema.common.SchemaImpl.validateSync(SchemaImpl.java:128) at io.vertx.json.schema.common.PropertiesValidatorFactory$PropertiesValidator.validateAsync(PropertiesValidatorFactory.java:166) at io.vertx.json.schema.common.SchemaImpl.runAsyncValidators(SchemaImpl.java:173) at io.vertx.json.schema.common.SchemaImpl.validateAsync(SchemaImpl.java:121) at io.vertx.json.schema.common.SchemaImpl.validateAsync(SchemaImpl.java:48) at validation.JsonValidationUtils.validateObject(JsonValidationUtils.java:60)

schema:

{ "description": "TS29510_Nnrf_NFManagement.yaml", "javaType": "ts3gpp.ChfInfo", "type": "object", "properties": { "supiRangeList": { "type": "array", "default": null, "minItems": 1, "items": { "$ref": "supi-range.json" } }, "gpsiRangeList": { "type": "array", "default": null, "minItems": 1, "items": { "$ref": "identity-range.json" } }, "plmnRangeList": { "type": "array", "default": null, "minItems": 1, "items": { "$ref": "plmn-range.json" } }, "groupId": { "type": "string" }, "primaryChfInstance": { "type": "string", "pattern": "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$" }, "secondaryChfInstance": { "type": "string", "pattern": "^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$" } }, "required": [ "primaryChfInstance", "secondaryChfInstance" ] }

java.util.ConcurrentModificationException at java.base/java.util.HashMap.forEach(HashMap.java:1339) at io.vertx.json.schema.common.SchemaRouterImpl.lambda$getScopeParentAliases$10(SchemaRouterImpl.java:215) at java.base/java.util.ArrayList.forEach(ArrayList.java:1541) at io.vertx.json.schema.common.SchemaRouterImpl.getScopeParentAliases(SchemaRouterImpl.java:215) at io.vertx.json.schema.common.SchemaRouterImpl.resolveAbsoluteUriAlternatives(SchemaRouterImpl.java:227) at io.vertx.json.schema.common.SchemaRouterImpl.resolveParentNode(SchemaRouterImpl.java:245) at io.vertx.json.schema.common.SchemaRouterImpl.resolveCachedSchema(SchemaRouterImpl.java:65) at io.vertx.json.schema.common.SchemaRouterImpl.resolveRef(SchemaRouterImpl.java:105) at io.vertx.json.schema.common.RefSchema.validateAsync(RefSchema.java:64) at io.vertx.json.schema.common.ItemsValidatorFactory$ItemsValidator.validateAsync(ItemsValidatorFactory.java:63) at io.vertx.json.schema.common.SchemaImpl.runAsyncValidators(SchemaImpl.java:173) at io.vertx.json.schema.common.SchemaImpl.validateAsync(SchemaImpl.java:121) at io.vertx.json.schema.common.PropertiesValidatorFactory$PropertiesValidator.validateAsync(PropertiesValidatorFactory.java:171) at io.vertx.json.schema.common.SchemaImpl.runAsyncValidators(SchemaImpl.java:173) at io.vertx.json.schema.common.SchemaImpl.validateAsync(SchemaImpl.java:121) at io.vertx.json.schema.common.SchemaImpl.validateAsync(SchemaImpl.java:48)

schema:
{ "description": "TS29510_Nnrf_NFManagement.yaml", "javaType": "ts3gpp.LmfInfo", "type": "object", "properties": { "externalClientType": { "type": "array", "default": null, "items": { "$ref": "enums/external-client-type.json", "existingJavaType": "ts3gpp.enums.ExternalClientType" } }, "lmfId": { "type": "string" }, "servingAccessTypes": { "type": "array", "default": null, "minItems": 1, "items": { "$ref": "enums/access-type.json", "existingJavaType": "ts3gpp.enums.AccessType" } }, "servingAnNodeTypes": { "type": "array", "default": null, "minItems": 1, "items": { "$ref": "enums/an-node-type.json", "existingJavaType": "ts3gpp.enums.AnNodeType" } }, "servingRatTypes": { "type": "array", "default": null, "minItems": 1, "items": { "$ref": "enums/rat-type.json", "existingJavaType": "ts3gpp.enums.RatType" } } } }

With this cleanup we will prepare vertx-json-schema for the Vert.x 5 release.

We will remove:

• deprecated classes:

  • Draft7SchemaParser
  • Draft201909SchemaParser
  • OpenAPI3SchemaParser
  • SchemaParser
  • SchemaRouter
  • Schema
  • SchemaRouterOptions

• classes that are only related to one the deprecated classes.

In openapi v3.0, the documented way to support null is with nullable: true. That doesn't seem to be supported by Vertx's json schema validation. This also will change in openapi v3.1.

https://www.openapis.org/blog/2021/02/16/migrating-from-openapi-3-0-to-3-1-0

Version

It affects vertx 4.4.5.
Looking at the code 5.x looks to not support this format too.

Context

"duration" is a defined JSON Schema format for string.
Using the vertx-web-openapi project, if the OpenApi file defined a field of type string and format duration the router builder fails to validate the field.

Version

4.3.4

Context

The code analysis tool “SpotBugs 4.7.3” pointed the following information out for further development considerations according to the implementation of the methods “validateAsync” and “validateSync” within the class “PropertiesValidator”.

💭 Inefficient use of keySet iterator instead of entrySet iterator