Patches to test cases about tck HOT 4 CLOSED

Hi @dmntk - I'm not sure what you're looking at achieve. The TCK tests are not incorrect and certainly are suitable for different test runners in other languages.

Some of the spec may be fairly Java-centric (IMO), but that does not stop a custom test runner (in any language) reading TCK tests and executing them against a runtime. For example, my test runner is written in Typescript and runs on nodejs using 'jest' as the backbone of the runner. It runs all tests. It is not Java.

A lot of the test patches I see are either decimals points related, or canonical strings related. So, let's discuss those.

Regarding decimal points - this is a 'by convention' thing AFAIK. In terms of equality checking between two decimals in your runtime, limit the precision to which your number equality comparison is based. Have a look at the drools source code for an example. So, two numbers might not be strictly equal, but if they are equal up to (if I recall) the ninth decimal place - with some rounding, then they are 'equal enough'. So, have your runtime 'back off' strict equality for decimals and go for the 'by convention' equality. There really is no point in us all chasing our tails over differences in big decimal implementations at the 30th decimal point. IMO, the practical considerations here are fair enough - business calculations rarely need equality at (say) 30 decimal places. If you feel your use cases need that strict precision, then consider making it an execution option for your runtime.

Re the 'canonical' strings. We should not confuse 'input data' confused with 'output data'. There are many ways to express a zero duration as input data. Many. Those TCK tests are valid and are exercising a runtime's ability to parse and deal with some of those as input data, and ultimately treat them as zero during execution. As there is (AFAIK) no canonical output ISO format for a zero duration, we've landed on just using one. Again - 'by convention'. But, 'by convention' is okay by me if it gets us all operating the same. IMO the spec needs to be updated to define the string representations of data.

If we alter those tests to all have the same format of zero duration input then we've lost coverage in the tests. We should probably have more tests with varied forms of zero to increase coverage, not less.

If other runtimes/runners are okay, I guess you need to ask yourself why you feel the TCK need to be altered, rather than the other way around.

There are some other things to take into account you may want to consider. Like, my compiler is very thorough and 'strict'. Many things like type differences, null coersion, unused requirements, missing expression logic and so on would normally emit an 'error' diagnostic - after all, the best runtime error is no runtime error!

.... but, the spec does not even contain the word 'compiler' .... so ...to be able to run the TCK tests - where we test lot a of runtime scenarios like null-coercion and type differences, I need to relax those settings to emit 'warning' diagnostics and let them be handled at runtime.

This means that, for the TCK tests, the compiler has to let through known error conditions and just 'deal with it and keep going'. And, on top of that, the runtime needs to be able to deal with things that, in theory, the compiler could have picked up as an error. One way I handle that is that when the compiler detects an error condition (say) in an expression, it substitutes a pre-cooked 'null' expression that returns null and emits a null-coercion 'error' diagnostic when executed. That way, the runtime does not have to perform the same checks, it simple runs the expression and gets a null return value. In AST terms, the compiler substitutes a specific type of node.

With strict mode' on, I doubt I could run half the TCK tests.

You also need to consider what an 'error' means and how you detect it. The spec does not cover errors, but, in the TCK, a lot of the tests have a flag saying they expect an error. What is that error? Well, it just means that the runtime returned a result but encountered an 'error' during execution - mostly that means a null-coercion situation as that is how bad stuff is handled generally. it is not necessary to pass the TCK tests, you kind of gives an indication that the test was expecting some error condition to be encountered and handled.

So, apols for the waffle, but I guess the point is, my feeling is you need to adapt your runtime and runner to be able to pass the tests, not the other way around.

Apols, I have not looked at every requested file patch, but, unless there is something very specific in there you feel is incorrect, I would say we leave the tests as they are.

from tck.

Hi @StrayAlien, I have already adopted my runner to pass the TCK tests. Thanks.

from tck.

cool. well done. Apols if I sounded preachy or anything, just trying to help.

from tck.

It's perfectly fine. I am glad that I am not the only one that struggles with the spec inconsistencies, decimals, canonicals, 'by conventions', coersions, infallible runtime errors and runners ;-). Take care :-)

from tck.