Comments (6)
Hi @Altai-man
Thanks for reporting the issue.
From what I remember, there was never a formal agreement on subtests. It appeared around the time TAP 13 was being submitted for the IETF as a draft. Some tools implemented it in a way, and others in another different way.
If memory serves me right, there used to be another older page in testanything.org, but it has been archived somewhere after the new site.
Take a look at this Perl doc: https://perldoc.perl.org/Test/More.html
Test::More is much older than node-tap, and probably used the old discussion/spec to implement it. In their example, if I understand correctly, the subtest follows whichever TAP Test Result that precedes it.
It is the first time I see brackets used in TAP. Before I had seen comments with #
being used as extra information about tests (metadata), as well as the YAMLish syntax. But anything else - including the brackets - would be discarded (which made the TAP syntax a bit trickier to be parsed due to ambiguity).
So not too sure how to proceed. I would like to adapt tap4j to support what node-tap expects, but I think we would have to wait for TestAnything/testanything.github.io#36 to be completed (TAP 14). Hopefully there will be standardization on subtests there.
In the meantime, I would prefer to follow what Perl, C, Shell, PHP, MySQL support (if I remember well the producer/consumers I used in these langs were based on Perl). Hope that makes sense.
from tap4j.
Thanks for you answer, @kinow!
To be honest, I actually want to parse TAP generated by Perl 6, so I took the node link just because it was close enough to refer to.
But, hmm, I may terribly misunderstood something right now, but on Test::More page I see this:
pass("First test");
subtest 'An example subtest' => sub {
plan tests => 2;
pass("This is a subtest");
pass("So is this");
};
pass("Third test");
generates:
ok 1 - First test
# Subtest: An example subtest
1..2
ok 1 - This is a subtest
ok 2 - So is this
ok 2 - An example subtest
ok 3 - Third test
I.e. first ok
- is a separate test, then subtest
, and overall subtest result is a second ok
, not the opposite. So each subtest result is coming before(!) overall subtest result, not after.
I think there may be some odd misunderstanding between us and it will be great if we could clear up this one.
By the way, Perl 6 suite(that implements what Test::More
does) generates this kind of code(first each subtest, then overall subtest result) too.
from tap4j.
Oh, I probably misread it. Let me have another look at the Perl example 👍
I need to find the old pages I used while writing that part... memory tells me I had to link the subtests with the previous test result. But I rarely trust my memory.
And thanks for pointing it out so politely my mistake :-)
from tap4j.
Oh well, the link that I sent before was saying the same as you pointed out. My mind was clearly showing me what I wanted :)
I've read some links from Perl, and everything points to the same way; i.e. subtests before. And it even makes sense for me now, as TAP is supposed to be able to be parsed in a Stream, continuously. So the indentation alone should be able to tell the parser it's a subtest... and as the number of test results may be omitted if I recall correctly, the subtests going before would be simpler to parse.
I really can't recall why I implemented it the other way. Will have to have another look later at the git commit history, and try again to find any links to old pages.
But said that, I think it's important to support the Perl behaviour, which would mean supporting node-tap as well :)
Sorry for the confusion @Altai-man
B
from tap4j.
No problem!
I am happy as long as initial Perl(and so C, MySQL et cetera) behavior will be parsed too at least as a feature with an additional parameter or something like that. 👍
from tap4j.
Closed with the related PR.
from tap4j.
Related Issues (20)
- Add Travis CI
- Update the code for Java 8
- Add CNAME HOT 1
- Error parsing TAP Stream: Invalid indentation HOT 18
- JUnit Test Runner doesn't output with Gradle HOT 1
- Junit TapListener method list bug HOT 4
- Fix maven site issues before next release HOT 1
- tap4j 4.4 missing from maven central HOT 6
- regression in subtest parsing between 4.2.1 an 4.3 HOT 7
- Subtests order/format change as of v4.3
- tap4j.org has been hacked HOT 1
- Move from multi-module to single-module project
- Unify changes report files HOT 1
- Fix build on Java > 8 HOT 4
- Fix apache-rat errors
- fix checkstyle errors HOT 3
- Find a home for old JUnit, TestNG, and JMeter integrations
- Update website
- Update to TAP 14
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tap4j.