leafle / newman-reporter-teamcity Goto Github PK
View Code? Open in Web Editor NEWA newman reporter for TeamCity.
License: MIT License
A newman reporter for TeamCity.
License: MIT License
any usage instructions would be very handy
CLI reporter is dominant. It means if there are two dominant reporters they will fight against each other and make output almost useless.
// Mark the CLI reporter as dominant, so that no two dominant reporters are together
PostmanCLIReporter.prototype.dominant = true;
I can't use -r cli,teamcity
together - common output is awful. I need teamcity diagnostics (start/stop/success/failure) in 'real time' while everything else I can read from cli
output in a pretty form.
In my TeamCity Build logs, I see the following:
##teamcity[testStarted name='xxxxTest Namexxxx' captureStandardOutput='true']
[11:41:22]
Incorrect property name.
Valid property list format is (name( )=( )'escaped_value'( )) where escape symbol is "|"
I was using newman and newman-reporter-teamcity in Powershell to run the API testing.
The newman command line is :
newman run $DirectAGCollectionURL -e $GetEnvVarAUTest --iteration-data $DataFile -n $Iterations --folder $APIFolderNames[$i] -r cli,teamcity,json --reporter-json-export $OutputFile
Below is the just copy & past of small part of the test report generated
...
└ Create Semaphore
##teamcity[testStarted name='Create Semaphore' captureStandardOutput='true']
POST https://au-test.leap.services/api/v1/semaphore [201 Created, 540B, 1028ms]
┌
│ 'Scenario 'No Semaphore Name' responseCode.code=201'
│ 'Defect API-18 is still outstanding'
└
√ 1) Check Expected RC = 201
√ 2) Validate response against schema
3. 3) Check SemaphoreId is NOT Created for 'No Semaphore Name' (Negative Scenario)
√ 4) Content-Type is application/json
##teamcity[testFailed name='Create Semaphore' message='3) Check SemaphoreId is NOT Created for |'No Semaphore Name|' (Negative Scenario)' details='3) Check SemaphoreId is NOT Created for |'No Semaphore Name|' (Negative Scenario) - Response code: 1028, reason: Created']
##teamcity[testFinished name='Create Semaphore' duration='1028']
Here the problem is, in the team city report, it gives wrong Response code by referring to the response time value(1028), not the real response code for the API (RC=201) .
The expectation is Response code: 201
My testing environment version:
In a command prompt window, type "npm list -g --depth=0 " to get some of the details related to newman:
+-- [email protected]
+-- [email protected]
First off, thank you for creating such an awesome Newman extension. We've been using it for quite a while.
When we run test cases in parallel, sometimes TeamCity will mismatch 'test case 1' to 'test suite 2'
##teamcity[testSuiteStarted name='test suite 1']
##teamcity[testSuiteStarted name='test suite 2']
##teamcity[testStarted name='test case 1' captureStandardOutput='true']
##teamcity[testFinished name='test case 1' duration='100']
##teamcity[testStarted name='test case 2' captureStandardOutput='true']
##teamcity[testFinished name='test case 2' duration='100']
##teamcity[testSuiteFinished name='test suite 2']
TeamCity would treat both 'test case 1' and 'test case 2' as part of 'test suite 2'.
Use flowId
I have given it a go. It works.
But I'm not happy with the fact that I required a third party library 'uuid/v4' to get the unique id.
Any thoughts on that?
Cheers,
following there instructions:
https://github.com/leafle/newman-reporter-teamcity
Tried to run my collection as per the instructions and got the following error:
warning: "teamcity" reporter could not be loaded.
teamcity was installed as per the instructions, so why can't it find it?
Note I'm using windows 10.
Any tests which are marked as Skipped in Postman collection and when running these test using Newman on TeamCity pass the tests, this creates confusion in the overall result.
Please suggest if there is a workaround to handle this case or kindly provide a fix.
Right now it's not obvious in the Teamcity UI what directory a Postman test is in a certain group or directory. It only reports the test name. The workaround is to include the directory name in the test name.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.