Comments (15)
Hi @rafaelcs ,
What do you exactly mean? Does it run alle the spec / step files again, including the spec / step-files that hold no failed tests?
protractor-flake can only run a spec / stepfile which holds at least 1 failed test. It can not specifically run the failed test.This due to the log parser that will check which test from which file failed, it will then collect all the files with failed specs and run them again, leaving out the spec / step-files that don't hold any failed tests.
from protractor-flake.
from protractor-flake.
Can you provide us the following info:
- configuration, how do you run protractor-flake
- logging
Tnx in advance
from protractor-flake.
Sorry for delay @wswebcreation
My flake file:
#!/usr/bin/env node
var protractorFlake = require('protractor-flake');
// skip first two passed args (node and self)
var protractorArgs = process.argv.splice(2);
protractorFlake({
protractorPath: 'node_modules/.bin/protractor',
maxAttempts: 2,
parser: 'standard',
nodeBin: 'node',
protractorArgs: protractorArgs
}, function(status, output) {
process.exit(status);
});
My conf.js
exports.config = {
directConnect: true,
specs: ['tests/UI/specs/*.js'],
seleniumAddress: 'http://localhost:4445/wd/hub',
framework: 'jasmine2',
onPrepare: function(){
// set implicit wait times in ms...
browser.manage().timeouts().implicitlyWait(5000);
browser.ignoreSynchronization = true;
// set browser size...
browser.manage().window().maximize();
// better jasmine 2 reports...
let SpecReporter = require('jasmine-spec-reporter').SpecReporter;
jasmine.getEnv().addReporter(new SpecReporter({displayStacktrace: 'spec'}));
},
capabilities: {
browserName: 'chrome',
restartBrowserBetweenTests: true
},
jasmineNodeOpts: {
showColors: true,
displayStacktrace: true,
displaySpecDuration: true,
// overrides jasmine's print method to report dot syntax for custom reports
print: function () {},
defaultTimeoutInterval: 999999
}
};
On this screen, you can see that only two tests failed
But when the tests ran again, all the specs were executed instead of just running the specs that failed
I'm running the tests using ./flake conf.js
That's my problem so far
from protractor-flake.
Hi @rafaelcs,
Is this the only trace you have from your tests, because I think the specreporter is the problem here. What protractor-flake does is that it:
- checks the exitcode after all tests are ran
- reads the log and parses it.
When I look at you log there is no logging that matches this regex of the standard parser.
I think there are 3 options for you:
- write a custom parser to use and parse your output
- change the logging of the specreporter so it will output more logging that will match the parser
- write a "hook" that will print the filename after a test has failed so it will be picked up by the standard / custom parser
Hope it helps
Grtz
Wim
from protractor-flake.
Recently I am also getting this error.
98 specs, 1 failure
Finished in 446.454 seconds
[21:37:14] I/launcher - 0 instance(s) of WebDriver still running
[21:37:14] I/launcher - chrome #1 failed 1 test(s)
[21:37:14] I/launcher - overall: 1 failed spec(s)
Closing report
[21:37:14] E/launcher - Process exited with error code 1
Using standard to parse output
Re-running tests: test attempt 2
Tests failed but no specs were found. All specs will be run again.
This is very frustrating as my all specs are running again even there is only one spec failure. Here is my configuration....
===gulpfile.js ===
gulp.task('e2e', function(){
return protractorFlake({
protractorPath: 'node_modules/protractor/bin/protractor',
maxAttempts: 3,
parser: 'standard',
// expects node to be in path
// set this to wherever the node bin is located
nodeBin: 'node',
// set color to one of the colors available at 'chalk' - https://github.com/chalk/ansi-styles#colors
color: 'magenta',
protractorArgs: ["e2e/conf/protractor.conf.js"]
}, function (status, output) {
process.exit(status);
});
});
==== package.json=====
"jasmine-core": "^2.6.0",
"protractor": "^5.1.1",
protractor-jasmine2-screenshot-reporter": "^0.3.5",
"protractor-flake": "2.5.1"
===protractor.conf.js=====
var HtmlScreenshotReporter = require('protractor-jasmine2-screenshot-reporter');
var htmlReporter = new HtmlScreenshotReporter({
dest: 'e2e/testreports',
filename: 'e2eReport.html',
reportOnlyFailedSpecs: false,
captureOnlyFailedSpecs: true,
showSummary: true,
showQuickLinks: true
});
exports.config = {
.......
specs: ['../spec/*.js'],
framework:'jasmine2',
multiCapabilities: [
{browserName: 'chrome'}
],
directConnect: true,
onPrepare: function() {
.......
jasmine.getEnv().addReporter(htmlReporter);
var jasmineReporters = require('jasmine-reporters');
jasmine.getEnv().addReporter(new jasmineReporters.JUnitXmlReporter({
consolidateAll: true,
savePath: 'e2e/testreports',
filePrefix: 'e2eReport'
}));
},
..........
And I will run gulp task to call protractor flake. Would you please check this one what is happening?
from protractor-flake.
Hi @surendraJ
Can you please provide your logging so we can see what protractor-flake should parse?
Tnx in advance
from protractor-flake.
@wswebcreation How can I generate the log? Is there any configuration do I need to fix to see the log?
from protractor-flake.
@surendraJ It's you console log / output from your protractor tests
from protractor-flake.
@wswebcreation here is my log I am not sure it would be helpful or not...
Stack:
Error: Failed expectation
at C:\Jenkins\workspace\mypro\e2e\spec\landing-page.spec.js:39:31
at ManagedPromise.invokeCallback_ (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:1366:14)
at TaskQueue.execute_ (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:2970:14)
at TaskQueue.executeNext_ (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:2953:27)
at asyncRun (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:2813:27)
at C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:676:7
at process._tickCallback (internal/process/next_tick.js:103:7)
98 specs, 1 failure
Finished in 446.454 seconds
[21:37:14] I/launcher - 0 instance(s) of WebDriver still running
[21:37:14] I/launcher - chrome #01 failed 1 test(s)
[21:37:14] I/launcher - overall: 1 failed spec(s)
Closing report
[21:37:14] E/launcher - Process exited with error code 1
Using standard to parse output
Re-running tests: test attempt 2
Tests failed but no specs were found. All specs will be run again.
from protractor-flake.
@surendraJ Is this the full log?
We need to the full log because, as also mentioned above, protractor-flake reads the output from protractor to determine which steps-files failed. Based on your logging I can check what goes wrong, so it would be nice if you can provide the complete output from your tests.
from protractor-flake.
@wswebcreation It's very huge log file so all are the success log before this one and failure message and again start 2nd attempt...
Failures:
1) #Landing page - [landing-page.spec.js] should show autocomplete with first entry of ZIP, 52 when entered Zip in location field
Message:
[31m Expected [ 'ZIP, 51', 'ZIP, 52', 'ZIP, 53', 'ZIP, 54', 'ZIP, 55', 'ZIP, 56' ] to contain 'ZIP, 61'.�[0m
Stack:
Error: Failed expectation
at C:\Jenkins\workspace\mypro\e2e\spec\landing-page.spec.js:38:31
at ManagedPromise.invokeCallback_ (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:1366:14)
at TaskQueue.execute_ (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:2970:14)
at TaskQueue.executeNext_ (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:2953:27)
at asyncRun (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:2813:27)
at C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:676:7
at process._tickCallback (internal/process/next_tick.js:103:7)
Message:
�[31m Expected [ 'ZIP, 51', 'ZIP, 52', 'ZIP, 53', 'ZIP, 54', 'ZIP, 55', 'ZIP, 56' ] to contain 'ZIP, 61'.�[0m
Stack:
Error: Failed expectation
at C:\Jenkins\workspace\mypro\e2e\spec\landing-page.spec.js:39:31
at ManagedPromise.invokeCallback_ (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:1366:14)
at TaskQueue.execute_ (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:2970:14)
at TaskQueue.executeNext_ (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:2953:27)
at asyncRun (C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:2813:27)
at C:\Jenkins\workspace\mypro\node_modules\selenium-webdriver\lib\promise.js:676:7
at process._tickCallback (internal/process/next_tick.js:103:7)
98 specs, 1 failure
Finished in 446.454 seconds
[21:37:14] I/launcher - 0 instance(s) of WebDriver still running
[21:37:14] I/launcher - chrome #01 failed 1 test(s)
[21:37:14] I/launcher - overall: 1 failed spec(s)
Closing report
[21:37:14] E/launcher - Process exited with error code 1
Using standard to parse output
Re-running tests: test attempt 2
Tests failed but no specs were found. All specs will be run again.
Report destination: e2e\testreports\e2eReport.html
[21:37:14] W/driverProviders - Using driver provider directConnect, but also found extra driver provider parameter(s): seleniumServerJar
[21:37:14] I/launcher - Running 1 instances of WebDriver
[21:37:14] I/direct - Using ChromeDriver directly...
from protractor-flake.
If this is the only logging for the failure then the standard / multi parser will not detect any failures in your logging. This means it will rerun all your specs again.
My assumption is that the jasmine-reporter
is "cleaning" up the log the shown in the output. In my opinion this is the same problem @rafaelcs has and can only be fixed with a "customer" parser.
@NickTomlin , how do you think about this?
from protractor-flake.
Hi @wswebcreation -
@NickTomlin - fyi
Please see my total log as below. You should see out of 3 tests, 2 tests have passed and 1 test failed. Expecting protractor-flake to re run ONLY the failed test in 2nd attempt. However, it runs all 3 tests again. Can you please suggest a solution or let me know if i missed something.
Thanks in advance.
C:\Users\wilson.palkuri\Downloads\protractor-example-master\protractor-example-master>protractor-flake --maxAttempts=2 -- conf.js
[14:53:34] I/launcher - Running 1 instances of WebDriver
[14:53:45] I/testLogger -
------------------------------------
[14:53:45] I/testLogger - [chrome #01-0] PID: 11920
[chrome #01-0] Specs: C:\Users\wilson.palkuri\Downloads\protractor-example-master\protractor-example-master\specs\friendSpec.js
[chrome #01-0]
[chrome #01-0] [14:53:38] I/direct - Using ChromeDriver directly...
[chrome #01-0] [14:53:45] I/runnerCli - Cannot find module 'chance'
[14:53:45] I/testLogger -
[14:53:45] E/launcher - Runner process exited unexpectedly with error code: 1
[14:53:46] I/launcher - 1 instance(s) of WebDriver still running
.[14:53:56] I/testLogger -
------------------------------------
[14:53:56] I/testLogger - [chrome #01-1] PID: 10200
[chrome #01-1] Specs: C:\Users\wilson.palkuri\Downloads\protractor-example-master\protractor-example-master\specs\nonAngularLoginSpec.js
[chrome #01-1]
[chrome #01-1] [14:53:49] I/direct - Using ChromeDriver directly...
[chrome #01-1] Jasmine started
[chrome #01-1] No Angular PASS
[chrome #01-1]
[chrome #01-1] non-angular login
[chrome #01-1] √ should goto friend pages on successful login
[chrome #01-1]
[chrome #01-1] Executed 1 of 1 spec SUCCESS in 1 sec.
[14:53:56] I/testLogger -
[14:53:56] I/launcher - 1 instance(s) of WebDriver still running
..[14:54:22] I/testLogger -
------------------------------------
[14:54:22] I/testLogger - [chrome #01-2] PID: 152
[chrome #01-2] Specs: C:\Users\wilson.palkuri\Downloads\protractor-example-master\protractor-example-master\specs\qsSpec.js
[chrome #01-2]
[chrome #01-2] [14:54:00] I/direct - Using ChromeDriver directly...
[chrome #01-2] Jasmine started
[chrome #01-2]
[chrome #01-2] Quality Shepherd blog
[chrome #01-2] √ should display 5 posts per page
[chrome #01-2] √ should return search results
[chrome #01-2]
[chrome #01-2] Executed 2 of 2 specs SUCCESS in 17 secs.
[14:54:22] I/testLogger -
[14:54:22] I/launcher - 0 instance(s) of WebDriver still running
[14:54:22] I/launcher - chrome #01-0 failed with exit code: 1
[14:54:22] I/launcher - chrome #01-1 passed
[14:54:22] I/launcher - chrome #01-2 passed
[14:54:22] I/launcher - overall: 1 process(es) failed to complete
[14:54:22] E/launcher - Process exited with error code 100
Using standard to parse output
Re-running tests: test attempt 2
Tests failed but no specs were found. All specs will be run again.
[14:54:27] I/launcher - Running 1 instances of WebDriver
[14:54:38] I/testLogger -
------------------------------------
[14:54:38] I/testLogger - [chrome #01-0] PID: 8196
[chrome #01-0] Specs: C:\Users\wilson.palkuri\Downloads\protractor-example-master\protractor-example-master\specs\friendSpec.js
[chrome #01-0]
[chrome #01-0] [14:54:31] I/direct - Using ChromeDriver directly...
[chrome #01-0] [14:54:38] I/runnerCli - Cannot find module 'chance'
[14:54:38] I/testLogger -
[14:54:38] E/launcher - Runner process exited unexpectedly with error code: 1
[14:54:38] I/launcher - 1 instance(s) of WebDriver still running
.[14:54:49] I/testLogger -
------------------------------------
[14:54:49] I/testLogger - [chrome #01-1] PID: 13060
[chrome #01-1] Specs: C:\Users\wilson.palkuri\Downloads\protractor-example-master\protractor-example-master\specs\nonAngularLoginSpec.js
[chrome #01-1]
[chrome #01-1] [14:54:42] I/direct - Using ChromeDriver directly...
[chrome #01-1] Jasmine started
[chrome #01-1] No Angular PASS
[chrome #01-1]
[chrome #01-1] non-angular login
[chrome #01-1] √ should goto friend pages on successful login
[chrome #01-1]
[chrome #01-1] Executed 1 of 1 spec SUCCESS in 1 sec.
[14:54:49] I/testLogger -
[14:54:50] I/launcher - 1 instance(s) of WebDriver still running
..[14:55:18] I/testLogger -
------------------------------------
[14:55:18] I/testLogger - [chrome #01-2] PID: 9324
[chrome #01-2] Specs: C:\Users\wilson.palkuri\Downloads\protractor-example-master\protractor-example-master\specs\qsSpec.js
[chrome #01-2]
[chrome #01-2] [14:54:54] I/direct - Using ChromeDriver directly...
[chrome #01-2] Jasmine started
[chrome #01-2]
[chrome #01-2] Quality Shepherd blog
[chrome #01-2] √ should display 5 posts per page
[chrome #01-2] √ should return search results
[chrome #01-2]
[chrome #01-2] Executed 2 of 2 specs SUCCESS in 18 secs.
[14:55:18] I/testLogger -
[14:55:18] I/launcher - 0 instance(s) of WebDriver still running
[14:55:18] I/launcher - chrome #01-0 failed with exit code: 1
[14:55:18] I/launcher - chrome #01-1 passed
[14:55:18] I/launcher - chrome #01-2 passed
[14:55:18] I/launcher - overall: 1 process(es) failed to complete
[14:55:18] E/launcher - Process exited with error code 100
C:\Users\wilson.palkuri\Downloads\protractor-example-master\protractor-example-master>
from protractor-flake.
Hi @wpalkuri
Can you provide more info about the parser you use and the config you use?
When I look at this part of your log
[14:53:45] I/testLogger - [chrome #01-0] PID: 11920
[chrome #01-0] Specs: C:\Users\wilson.palkuri\Downloads\protractor-example-master\protractor-example-master\specs\friendSpec.js
[chrome #01-0]
[chrome #01-0] [14:53:38] I/direct - Using ChromeDriver directly...
[chrome #01-0] [14:53:45] I/runnerCli - Cannot find module 'chance'
[14:53:45] I/testLogger -
[14:53:45] E/launcher - Runner process exited unexpectedly with error code: 1
[14:53:46] I/launcher - 1 instance(s) of WebDriver still running
.[14:53:56] I/testLogger -
I see a problem in your code you don't want to rerun, but just fix ;-)
The reason why the parser (don't exactly know which one you use) is that no exception is thrown that could be picked up by the parser to determine which spec file to rerun. Are you using SpecReporter or an other tool to do the reporting? If so, try enabling the stacktrace option
from protractor-flake.
Related Issues (20)
- How to build with ng e2e and run config file via protractor-flake? HOT 7
- The automated release is failing 🚨 HOT 1
- Protractor-flake no show outputs in console HOT 18
- Reruning all the tests not just the failed ones with cucumber. HOT 1
- protractor-flake doesn't run tests if suite is not specified HOT 3
- protractor-flake doesn't run the remaining tests after retrying the failed test HOT 5
- protractor-flake finds the typescript file (not the compiled file) on retry HOT 5
- protractor flake executing all suites mentioned instead of specific suite HOT 2
- Feature Request: Option to mark failing and pending specs as failure to be picked up for rerun in flake
- Same failed spec being marked to run twice with cucumber HOT 3
- Failing Cucumber Scenarios causing entire feature files to re-run HOT 4
- Protractor Flake is not working with cucumber? HOT 2
- Feature Request: Expose attemptNo or allow code to be specified in beforeAttempts/afterAttempts HOT 6
- Easier way to specify path of custom parser HOT 5
- screenshots of retries overwrite the original run HOT 4
- Running serve as part of Flake HOT 4
- retryconfig doesn't work HOT 3
- running node ./protractor-flake-retry.js works putting than command inside npm run test fails HOT 3
- seleniumAddress HOT 2
- how to use protractor arguments with yarn HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from protractor-flake.