Add support for jasmine 2 about grunt-saucelabs HOT 29 OPEN

I've tried to get the ball rolling with jasmine-jsreporter and jasmine itself - it seems a bunch of information that used to be exposed to reporters in jasmine 1.3.1 is now no-longer available.

detro/jasmine-jsreporter#7

jasmine/jasmine#547

from grunt-saucelabs.

the Sauce api gets the JSON data from the old jsreporter and does a lot of parsing on that specific structure to get the data to populate the various fields Sauce uses. The original idea was that everyone already using Jasmine could easily expose their results and Sauce parses it to get what they need. But there's no consistent method for getting the results in Jasmine 2.0 yet.

My preference would be to use the custom framework, explained in the docs. Writing a small script to parse the results into that format would get you most of the data you need, correct? Other people can use the same, or a modified version of the script.

Are there more fields you'd like supported on the custom framework? Or would you prefer to have @sclevine's specific reporter added to sauce's framework support?

from grunt-saucelabs.

Ideally if @sclevine's reporter works, have that added to sauce's framework support for when jasmine version is set to 2.0.0.

I'll give custom a play in the coming weeks, see how far I get.

from grunt-saucelabs.

What's the status on this?

from grunt-saucelabs.

@rkh - I've made JSReporter with Jasmine 2.0, and have written a JSReporter2 class as part of that project.

Still waiting on Jasmine#575 - the Pivotal guys seem to be MIA for over a month now

from grunt-saucelabs.

For what it's worth, I solved this by using karma.

from grunt-saucelabs.

Update: pivotal have merged in the required reporting changes in jasmine 2.0.1 (jasmine/jasmine#575), and jasmine-jsreporter has been updated to match (detro/jasmine-jsreporter#17).

There should no-longer be anything blocking grunt-saucelabs from adopting Jasmine 2.0.1.

Jasmine 2.0.1 and jasmine-jsreporter need to be versioned up in grunt-saucelabs examples folder: https://github.com/axemclion/grunt-saucelabs/tree/b25ff61b7ce0126bbe507f931758ce8cfb64bae7/examples/jasmine/lib

Looks like @stdavis has opened gruntjs/grunt-contrib-jasmine#154 to progress this in grunt-contrib-jasmine

from grunt-saucelabs.

just got back from vacation, I'll see about adding this in.

from grunt-saucelabs.

👍 we would love to start getting saucy and I believe this is what we are waiting on.

from grunt-saucelabs.

thanks for pinging me @steveoh
I'll up the priority.

from grunt-saucelabs.

any updates @Jonahss?

from grunt-saucelabs.

Currently working on Appium, maybe if I get my tasks done quick enough I can sneak this in.

from grunt-saucelabs.

@Jonahss if you tell me what needs to happen I can try to do it.

from grunt-saucelabs.

Right now, if you run a unit test and pass in the jasmine framework, Sauce's servers expect to see results in the same format as the test results reporter by the old JasmineReporter.

If you run a nut test and pass in the custom framework, then Sauce's servers are looking for a json object which conforms to the custom framework as outlined in the README.

To add support for a new format, it requires that a Sauce employee add the capability to properly parse the results to the Sauce servers.
Unfortunately you can't help with that :( (unless you want a new job)

So your other option is to parse the jasmine2.1 results yourself and get them into either the custom or jasmineReporter formats already supported.

Thanks for continuing to keep this issue active.

from grunt-saucelabs.

So if you use jasmine-jsreporter and add JSReporter2 as follows:

jasmine.getEnv().addReporter(new jasmine.JSReporter2())

Results should automatically be populated in the old JasmineReporter format.

(Provided the jasmine 2 reporters haven't changed between 2.0.1 and 2.1)

from grunt-saucelabs.

So whats happening with this? Is this supported as long as I use the right reporter?

from grunt-saucelabs.

@lukeapage if you follow the instructions given by @alextreppass above, you should be able to use jasmine2.0

from grunt-saucelabs.

Thanks, I have done that, was just confused originally as to why this was open if its not an issue.

So far, no luck
https://saucelabs.com/jobs/26aa971edb9d4becb74b2c82edc51e86

https://travis-ci.org/less/less.js/jobs/38538939

https://github.com/less/less.js/pull/2236/files

It looks to me like status is blank? But everything else is there? Any suggestions?

from grunt-saucelabs.

@lukeapage you might be running into the issue where Sauce can't handle super long test results. Try with a smaller test suite.

from grunt-saucelabs.

I read it was 64kb and didn't think we had reached that, but I tried a smaller suite and it worked - thanks. Any idea if that limit will be lifted? Has anyone raised it with sauce?
p.s. great repo, it made things super easy

from grunt-saucelabs.

Thanks!
Most the credit due to all the great collaborators.

Sauce is aware of the limit issue, and it's slated to be fixed. Don't know
the deadline though.

On Mon, Oct 20, 2014 at 11:56 PM, Luke Page [email protected]
wrote:

I read it was 64kb and didn't think we had reached that, but I tried a
smaller suite and it worked - thanks. Any idea if that limit will be
lifted? Has anyone raised it with sauce?
p.s. great repo, it made things super easy

—
Reply to this email directly or view it on GitHub
#109 (comment)
.

from grunt-saucelabs.

Thanks for the helpful tips @Jonahss. I too ran into the larger test suite. I only have 185 specs which doesn't seem that much to me but running a subset of 14 made everything work. Do you have a link to the bug at sauce that we can track. This is a show stopper for me until I can run my entire test suite.

from grunt-saucelabs.

@stdavis it shouldn't be a show stopper. You can just alter your reporting script to only add the info for a failure, and not even bother reporting passes.

from grunt-saucelabs.

@Jonahss What do you mean by "reporting script"? You mean alter jasmine-jsreporter?

from grunt-saucelabs.

Ah yes, altering the jasmine reporter OR just override the call to the reporter and sanitize the results. Sauce Labs runs the following Javascript code to get the results from your page:
window.jasmine.getJSReport()

So if you override that method with your own, and call the original method to get and parse the results, you're done.
PSEUDO CODE:

var oldFunc = window.jasmine.getJSReport;
window.jasmine.getJSReport = function() {
 var results = oldFunc();
 return removePassingTests(results)
}

Also check out using the custom framework, to have even finer control over what gets reported on Sauce.

from grunt-saucelabs.

@Jonahss Thanks for the tip. I will implement the work around that you suggest. Really appreciate your help.

from grunt-saucelabs.

I've tried to implement your pseudo code and got it working. This is dependent on jquery

jasmine.getEnv().addReporter(new jasmine.JSReporter2());

(function () {
   var oldFunc = window.jasmine.getJSReport;
   window.jasmine.getJSReport = function () {
      var results = oldFunc();
      if (results) {
         return {
            durationSec: results.durationSec,
            suites: removePassingTests(results.suites),
            passed: results.passed
         };
      } else {
         return null;
      }
   };

   function removePassingTests(suites) {
      return $.map($.grep(suites, grepFailed), mapSuite);
   }

   function mapSuite(suite) {
      return $.extend({}, suite, {
         specs: $.grep(suite.specs, grepFailed),
         suites: removePassingTests(suite.suites)
      });
   }

   function grepFailed(item) {
      return !item.passed;
   }
})();

from grunt-saucelabs.

@alexan Thanks, You save my time.

from grunt-saucelabs.

FWIW, ES5-only no-jquery version of @alexan's solution for trimming the passing tests:

jasmine.getEnv().addReporter(new jasmine.JSReporter2());

(function () {
    var oldJSReport = window.jasmine.getJSReport;
    window.jasmine.getJSReport = function () {
        var results = oldJSReport();
        if (results) {
            return {
                durationSec: results.durationSec,
                suites: removePassingTests(results.suites),
                passed: results.passed
            };
        } else {
            return null;
        }
    };

    function removePassingTests (suites) {
        return suites.filter(specFailed)
            .map(mapSuite);
    }

    function mapSuite (suite) {
        var result = {};
        for (var s in suite) {
            result[s] = suite[s];
        }
        result.specs = suite.specs.filter(specFailed);
        result.suites = removePassingTests(suite.suites);
        return result;
    }

    function specFailed (item) {
        return !item.passed;
    }
})();

from grunt-saucelabs.