Expose specific reason for build failure about steps-xcode-test HOT 10 CLOSED

hi @eugeneshih - wouldn't this be better solved by providing a generic solution, in bitrise (https://github.com/bitrise-io/bitrise). Something like $BITRISE_FIRST_FAILED_STEP_TITLE - to get the title of the first failed step, and maybe a combined $BITRISE_FAILED_STEP_TITLES which would be a list of the failed step titles (separated by a ; character or something similar).

What do you think?

from steps-xcode-test.

That would be fine. 👍

from steps-xcode-test.

@eugeneshih could you please describe what exactly would you like to get - for what you would like to use the information? This would help us to finalize the specification for the feature.

from steps-xcode-test.

Currently, my curl step takes the value of $BITRISE_UNITTEST_STATUS and makes a post to a special HTTP server. It includes the result of $BITRISE_UNITTEST_STATUS as a parameter, for example:

-u [destination] -d "subject=$BITRISE_APP_TITLE/$BITRISE_GIT_BRANCH" -d "unit tests: **$BITRISE_UNITTEST_STATUS**"

Both suggestions you made seem useful, but the first would be most useful for me.

from steps-xcode-test.

What would you like to get @eugeneshih - the Title of the failed step, or rather the "success" / "error" state/text of a specific step? Keep in mind that in general if a Step before the Test step fails the Test step would not even performed (unless you mark it as is_always_run), so you might get other states than just "success" and "error" (namely: "skipped").

It would help us if you could share what exactly do you want to use the environment for - do you want to just send a message to somewhere with the title of the failed step? Or you need the state of a specific step?

from steps-xcode-test.

I was pretty happy with the old variable. So, in answer to your questions, I would most like to know the "success" / "failure" state of the unit test step. This status is simply sent via a curl POST call as I show above.

I don't know if I can be any clearer.

If you want to provide a more general reporting mechanism which includes the title of the failed step, that's be great, but I don't need that. If you want to provide a dictionary of steps mapped to their success / failure state, that would be great too. Clearly, those would be more useful, but I don't need those features at this point.

from steps-xcode-test.

Thanks for your feedback, it really helps us to figure out a generic solution, so that the individual steps don't have to implement a mechanism like this!

One last question: would you like a textual status ("success"/"failure") or a binary one (1/0)?

from steps-xcode-test.

Textual is better for now.

from steps-xcode-test.

@eugeneshih this will be available in the next version of the step, but it'll require a new bitrise CLI version as well if you would like to run it on your own Mac, as the current version does not expose step outputs in case the step fails. This'll be fixed in a new patch release of bitrise, and the VMs will be patched as well asap.

We'll release the new version of this step once everything else is released & deployed.

from steps-xcode-test.

@eugeneshih the new version is now available, once again thank you for your feedback! https://github.com/bitrise-io/steps-xcode-test/releases/tag/1.3.0

from steps-xcode-test.