SmalltalkCI>>readSTONSpec: is not portable to GemStone about smalltalkci HOT 31 CLOSED

Okay, what I thought was a move of the smalltalkCI code into packages "package support" was not what I thought it was ... you realize that I will have to hand edit these chunk files to avoid massive changes when filed out from GemStone :(

from smalltalkci.

I think I'll create a FileTree repo somewhere with this code in it and then let you create chunk files :)

from smalltalkci.

Hmmmm, I can't even edit the .st files with a test editor because of line ending issues (doing development on a mac) ....

from smalltalkci.

... I'll edit the chunk files in the web browser

from smalltalkci.

No problem at all, I'm happy to take care of the rest.
.st files aren't really nice to develop with, but they are easy to deploy since they don't require anything to be installed in an image.

from smalltalkci.

Thanks, I'm just about ready to load/run the test packages, which I presume are only intended for testing the SmalltalkCI implementation (they don't appear to be run during the actual travis-ci run from the test scripts although, I still can't tell what's actually being tested there:). Shouldn't these tests be run by travis (I can't tell if they are or not...)

I assume that there are at least a handful of additional changes that I will need to be make for portability... ...

from smalltalkci.

Yes, the Smalltalk tests in ./tests are self-tests and Travis loads and runs them when testing smalltalkCI itself. The code coverage isn't really high at the moment as the code is still changing a lot, but I hope this all will change soon.

from smalltalkci.

Are you referring to the tests in https://github.com/dalehenrich/smalltalkCI/tree/issue_48/tests/SmalltalkCI-Tests.package .... if so could you point me to where those tests are loaded?

Also these SmalltalkCI-Tests look like they are platform-specific ... perhaps even pharo-specific ... i would like to refactor things a bit so that the tests can be multi-platform ... Do they pass on Squeak?

from smalltalkci.

The loading and testing is specified in the .smalltalk.ston and yes, they do pass on Squeak. However, it totally makes sense to have platform-specific tests as well to test platform-specific code!

from smalltalkci.

Okey doke, that means that the tests that don't pass in GemStone are #squeakCommon tests and need to be packaged separately .... or .... it looks like SmalltalkCI platformSymbol is used to isolate some pharo-specific tests and right now the only test that is failing is SmalltalkCITest>>testClassesToTest, because the categories are specific to pharo/squeak ... so it looks like I won't have to split things up into separate packages quite yet ...

Of course I think I saw some additional code that shouldn't run on GemStone, so if indeed that's the case, I'll add addiitional test coverage:)

from smalltalkci.

Perfect!
Also, we are free to change the code at anytime as long as we don't break the API 😄
At some point, we do want platform-specific tests, but first things first: let's make GemStone run in a Travis container!

from smalltalkci.

I'm inclined to add SmalltalkCI class>>runCIForSpec: where you pass in a reified instance of SmalltalkCISpec to initiate tests ... if you don't want it in SmalltalkCI, I'll put it in SmalltalkCIGemstone.

from smalltalkci.

Don't worry too much about things like this. I will have to go over the code anyway once you are done and this way I might learn something new as well 😄

from smalltalkci.

well, it looks like using the web browser to edit the .st files converts all of the CRs to LFs and no good diffs ... CRs in the .st file is probably a bad idea if we want to be able to review changes on github

from smalltalkci.

see this diff

from smalltalkci.

@fniephaus I can't find the code that causes a run to fail ... is this some more magic from smalltalk.rb? If so what does it expect?

from smalltalkci.

These CRs are really annoying and I've already thought about reformatting the files after exporting with another editor to make them more git-compatible.

A build currently fails if:

• the loading or testing process closes the image with exit code !=0, or
• when there are any errors or failures in the JUnit XML

from smalltalkci.

Ah okay ... then I'll have run.sh check the results of the tests end exit accordingly .. thanks

from smalltalkci.

Is there any way GemStone can also export test results in an XML file?
smalltalkCI pretty-prints test results to the Travis log (example).

from smalltalkci.

Other than the fact that I just finished ripping out all of the xml writer code when I ported SCITestReporter?

That was the information I was missing ... that the xml output was required ... ah well ... I'll just see if I can restore the xml code ...

I won't get to see any of this pretty printing goodness for a while right, since we're still waiting for the gemstone stuff to be merged and then wait for it to be deployed?

from smalltalkci.

How is the error and stack information logged ... do you have a sample for that?

from smalltalkci.

Sorry, @dalehenrich. I thought it was clear that testing can finish successfully and produce a XML, even though there are tests failing. But this information is in the XML.

This is what the XML file looks like if I run all tests (including the excluded ones):

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="SmalltalkCI" tests="11" failures="2" errors="1" time="0.004">                                                               
    <testcase classname="SmalltalkCI.Tests.SCIAbstractLoadSpecTest" name="testIsComplete" time="0.0">
    </testcase>
    <testcase classname="SmalltalkCI.Excluded.Tests.SCIExcludedTests" name="testError" time="0.002">
        <error type="Error" message="An error message.">SCIExcludedTests(Object)>>error:
SCIExcludedTests>>testError
SCIExcludedTests(TestCase)>>performTest
</error>
    </testcase>
    <testcase classname="SmalltalkCI.Excluded.Tests.SCIExcludedTests" name="testFailure" time="0.0">
        <failure type="TestFailure" message="A failure message.">SCIExcludedTests(TestCase)>>assert:description:
SCIExcludedTests(TestCase)>>fail:
SCIExcludedTests>>testFailure
SCIExcludedTests(TestCase)>>performTest
</failure>
    </testcase>
    <testcase classname="SmalltalkCI.Excluded.Tests.SCIExcludedTests" name="testShouldFail" time="0.0">
    </testcase>
    <testcase classname="SmalltalkCI.Excluded.Tests.SCIExcludedTests" name="testShouldPass" time="0.0">
    </testcase>
    <testcase classname="SmalltalkCI.Excluded.Tests.SCIExcludedTests" name="testShouldPassUnexpectedly" time="0.0">
        <failure type="UnexpectedTestPass" message="Test passed unexpectedly."></failure>
    </testcase>
    <testcase classname="SmalltalkCI.Tests.SCIMetacelloLoadSpecTest" name="testIsComplete" time="0.0">
    </testcase>
    <testcase classname="SmalltalkCI.Tests.SCIMetacelloLoadSpecTest" name="testIsPlatformCompatible" time="0.0">
    </testcase>
    <testcase classname="SmalltalkCI.Tests.SmalltalkCISpecTest" name="testAddSpecs" time="0.0">
    </testcase>
    <testcase classname="SmalltalkCI.Tests.SmalltalkCITest" name="testClassesToTest" time="0.0">
    </testcase>
    <testcase classname="SmalltalkCI.Tests.SmalltalkCITest" name="testRequireSTON" time="0.0">
    </testcase>
    <system-out><![CDATA[]]></system-out>
    <system-err><![CDATA[]]></system-err>
</testsuite>

from smalltalkci.

Yea, I thought the xml was there for the Jenkins/Hudsons jobs and not for SmalltalkCI ...

I was curious what the travis-ci output would look like in the case of an error ... I am curious what a failed test output looks like .... since I won't actually be able to see any of this for myself for several weeks ...

from smalltalkci.

What directory is the xml file supposed to show up in?

from smalltalkci.

Just so you know, I think it is worth the effort to produce the xml log file :)

from smalltalkci.

Looking at the failure of a recent GemStone run:

Creating starter scripts pharo and pharo-ui
On a 64-bit system? You must install the 32-bit libraries
   Try `sudo aptitude install ia32-libs` or see http://pharo.org/gnu-linux-installation#64-bit-System-Setup for more info
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  2771  100  2771    0     0  16847      0 --:--:-- --:--:-- --:--:-- 33792
Downloading the latest 30 Image:
    http://files.pharo.org/get-files/30/pharo.zip

Pharo.image
The 32-bit libraries required by Pharo have not been installed on this system.

The failure is due to the fact that the pharo download is failing because of the lack of 32-bit libraries installed ... but since you are able to run 32-bit pharo vms in continainers, I don't understand why this would cause GemStone to fail .... the same libs are needed ... unless there are other failures that you are trying to address that I am not aware of ...

from smalltalkci.

I currently don't have an example at hand, but failing tests are basically marked orange with an orange X, errors are marked red with a red X (just as in the Test Runner) and the corresponding error message is displayed in a Travis fold.

The reason, why you get those Pharo errors atm is that Travis does not install any dependencies at them moment at all when running a GemStone build. I have fixed that problem in travis-ci/travis-build#607.
However, you could run those builds on GCE and install all dependencies using GsDevKit again for the time being. I am not sure if that is worth the work to be honest.

Great that you are pro xml!
The XML file is supposed to be created in the build folder which is the folder the image is in.

from smalltalkci.

Yeah, but I don't know "what folder the image is in" :) .... is $SMALLTALK_CI_HOME the right directory ... there are several different env vars floating around, but the only one that matters is the directory where the xml file is expected to be ...

from smalltalkci.

You are looking for $SMALLTALK_CI_BUILD :)

from smalltalkci.

THX!

from smalltalkci.

I will close this in favor of #51

from smalltalkci.