Comments (4)
Hi @dmntk - I'm not sure what you're looking at achieve. The TCK tests are not incorrect and certainly are suitable for different test runners in other languages.
Some of the spec may be fairly Java-centric (IMO), but that does not stop a custom test runner (in any language) reading TCK tests and executing them against a runtime. For example, my test runner is written in Typescript and runs on nodejs using 'jest' as the backbone of the runner. It runs all tests. It is not Java.
A lot of the test patches I see are either decimals points related, or canonical strings related. So, let's discuss those.
Regarding decimal points - this is a 'by convention' thing AFAIK. In terms of equality checking between two decimals in your runtime, limit the precision to which your number equality comparison is based. Have a look at the drools source code for an example. So, two numbers might not be strictly equal, but if they are equal up to (if I recall) the ninth decimal place - with some rounding, then they are 'equal enough'. So, have your runtime 'back off' strict equality for decimals and go for the 'by convention' equality. There really is no point in us all chasing our tails over differences in big decimal implementations at the 30th decimal point. IMO, the practical considerations here are fair enough - business calculations rarely need equality at (say) 30 decimal places. If you feel your use cases need that strict precision, then consider making it an execution option for your runtime.
Re the 'canonical' strings. We should not confuse 'input data' confused with 'output data'. There are many ways to express a zero duration as input data. Many. Those TCK tests are valid and are exercising a runtime's ability to parse and deal with some of those as input data, and ultimately treat them as zero during execution. As there is (AFAIK) no canonical output ISO format for a zero duration, we've landed on just using one. Again - 'by convention'. But, 'by convention' is okay by me if it gets us all operating the same. IMO the spec needs to be updated to define the string representations of data.
If we alter those tests to all have the same format of zero duration input then we've lost coverage in the tests. We should probably have more tests with varied forms of zero to increase coverage, not less.
If other runtimes/runners are okay, I guess you need to ask yourself why you feel the TCK need to be altered, rather than the other way around.
There are some other things to take into account you may want to consider. Like, my compiler is very thorough and 'strict'. Many things like type differences, null coersion, unused requirements, missing expression logic and so on would normally emit an 'error' diagnostic - after all, the best runtime error is no runtime error!
.... but, the spec does not even contain the word 'compiler' .... so ...to be able to run the TCK tests - where we test lot a of runtime scenarios like null-coercion and type differences, I need to relax those settings to emit 'warning' diagnostics and let them be handled at runtime.
This means that, for the TCK tests, the compiler has to let through known error conditions and just 'deal with it and keep going'. And, on top of that, the runtime needs to be able to deal with things that, in theory, the compiler could have picked up as an error. One way I handle that is that when the compiler detects an error condition (say) in an expression, it substitutes a pre-cooked 'null' expression that returns null and emits a null-coercion 'error' diagnostic when executed. That way, the runtime does not have to perform the same checks, it simple runs the expression and gets a null return value. In AST terms, the compiler substitutes a specific type of node.
With strict mode' on, I doubt I could run half the TCK tests.
You also need to consider what an 'error' means and how you detect it. The spec does not cover errors, but, in the TCK, a lot of the tests have a flag saying they expect an error. What is that error? Well, it just means that the runtime returned a result but encountered an 'error' during execution - mostly that means a null-coercion situation as that is how bad stuff is handled generally. it is not necessary to pass the TCK tests, you kind of gives an indication that the test was expecting some error condition to be encountered and handled.
So, apols for the waffle, but I guess the point is, my feeling is you need to adapt your runtime and runner to be able to pass the tests, not the other way around.
Apols, I have not looked at every requested file patch, but, unless there is something very specific in there you feel is incorrect, I would say we leave the tests as they are.
from tck.
Hi @StrayAlien, I have already adopted my runner to pass the TCK tests. Thanks.
from tck.
cool. well done. Apols if I sounded preachy or anything, just trying to help.
from tck.
It's perfectly fine. I am glad that I am not the only one that struggles with the spec inconsistencies, decimals, canonicals, 'by conventions', coersions, infallible runtime errors and runners ;-). Take care :-)
from tck.
Related Issues (20)
- Clarify `null` handling in `FunctionInvocation` HOT 6
- Cover comparisons
- Create transitive import test cases based on snippets in DMN15-58 HOT 2
- Doubtful typing on overloaded built-in function in test HOT 2
- Report does not work under windows
- The test cases table breaks up around Compliance-Level-3
- New PRs ... but deleting them. The PRs have dragged in CL/LF differences HOT 2
- Confused about decision service encapsulatedDecisions - ambiguous - is ch11 example correct? HOT 5
- Task list for DMN 1.6
- Optional fn params vs null values HOT 4
- Question to test case `decision_015` in `0084-feel-for-loops.dmn` HOT 1
- Quesion to test case `nested009` in `1146-feel-context-put-function.dmn` HOT 2
- 0014-loan-comparison assumes that sorting keeps the original order HOT 2
- breaking change?: what do do with range equality? What is 'undefined'? HOT 4
- Is `[100] = 100` false or null? HOT 4
- Does @"P0D" = "P0Y" give `true` or `null` (or false?) ? HOT 3
- The new text for type conversions in "10.3.2.9.4 Type conversions" misses some use cases HOT 2
- Spec permits ranges to use new `=` and `!=` comparison operators - but it doesn't make much sense (to me) HOT 3
- TCK is using yearMonthDuration/dayTimeDuration/dateTime typeRefs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tck.