Can you expose the xccovreport file ? about steps-xcode-test HOT 20 CLOSED

So we got a version of this working in our Bitrise builds for Xcode 11+ specifically. Due to the new format of xcresult starting from Xcode 11, our intern made a tool called xcparse for extracting out the code coverage file from a given xcresult path (Bitrise's handy $BITRISE_XCRESULT_PATH). We then have a step where we install the Swift command line tool via Homebrew (important to disable the upgrade part since it doesn't play well with the tap install). At that point, we have the action.xccovreport in the source directory & use it with xcov.

From our Bitrise.yml

    - [email protected]:
        inputs:
        - upgrade: 'no'
        - packages: ChargePoint/xcparse/xcparse
        title: Brew install xcparse
        is_always_run: true
    - [email protected]:
        inputs:
        - content: |-
            #!/usr/bin/env bash

            # Extract Xcode 11 xcovreport
            echo $BITRISE_XCRESULT_PATH

            xcparse -x "$BITRISE_XCRESULT_PATH" .

            cp action.xccovreport $BITRISE_DEPLOY_DIR

            zip -r xcresult.zip $BITRISE_XCRESULT_PATH
            cp xcresult.zip $BITRISE_DEPLOY_DIR
        title: Extract xccovreport
        is_always_run: true
    - [email protected]:
        title: xcov
        inputs:
        - content: |-
            #!/usr/bin/env bash
            # debug log
            set -x

            bundle install
            bundle exec xcov -w $BITRISE_PROJECT_PATH -s $BITRISE_SCHEME -f action.xccovreport -o reports
            zip -r $BITRISE_DEPLOY_DIR/xcov_report.zip reports/*
        is_always_run: true

from steps-xcode-test.

Looks like xcov 1.7 was released and fixed the issue!

from steps-xcode-test.

@brandtdaniels In the steps I had above, you need to have the two steps before the xcov step - one that installs the ChargePoint xcparse tool via Homebrew & the second step to use xcparse to get the xccovreport out of the xcresult file. The log you gave seems to imply you don't have those two steps. If you do have those steps, see if you can attach the logs from those steps too.

That said, since I wrote that answer, Fastlane took ownership of xcov and their maintainer updated xcov so that the -f option can now take the xcresult directly (rather than having to use xcparse to get it for you). So you could change towards something like this:

bundle exec xcov -w $BITRISE_PROJECT_PATH -s $BITRISE_SCHEME -f $BITRISE_XCRESULT_PATH -o reports

from steps-xcode-test.

@brandtdaniels In the steps I had above, you need to have the two steps before the xcov step - one that installs the ChargePoint xcparse tool via Homebrew & the second step to use xcparse to get the xccovreport out of the xcresult file. The log you gave seems to imply you don't have those two steps. If you do have those steps, see if you can attach the logs from those steps too.

That said, since I wrote that answer, Fastlane took ownership of xcov and their maintainer updated xcov so that the -f option can now take the xcresult directly (rather than having to use xcparse to get it for you). So you could change towards something like this:

bundle exec xcov -w $BITRISE_PROJECT_PATH -s $BITRISE_SCHEME -f $BITRISE_XCRESULT_PATH -o reports

After so many different methods, bundle exec xcov -w $BITRISE_PROJECT_PATH -s $BITRISE_SCHEME -f $BITRISE_XCRESULT_PATH -o reports is what worked for me. Thanks!

from steps-xcode-test.

Hy @mackoj I will create a task for the tooling team with this, they will get back to you when they done it.

from steps-xcode-test.

Thanks @fehersanyi-bitrise today you allow us to create this file but the file path is not available in the output variable of the step.

from steps-xcode-test.

Yes I know what you mean, the file is there, but it would be easier to manage is the step exposed like a $XCCOV_REPORT_PATH variable or something with the path to the file 🙃

from steps-xcode-test.

Exactly ! 😁

from steps-xcode-test.

Is there any progression on this issue ? thanks

from steps-xcode-test.

Hello @icPJmobile
unfortunately the tooling team could not yet take up the task.

from steps-xcode-test.

Have you made some progress on this point @fehersanyi-bitrise ?

from steps-xcode-test.

Hi @mackoj!

This wasn't completed just yet, as promised we will definitely declare this on this thread once it is! Thanks for the patience 🙂

from steps-xcode-test.

This is really great @abotkin-cpi, thanks for sharing!

from steps-xcode-test.

It looks like xcov is broken with Xcode 11, so I wonder how this worked.
fastlane/fastlane#15295

The steps above gave me this output:

+ bundle exec xcov -w CRM.xcworkspace -s CRM -f action.xccovreport -o reports
[17:21:26]: Sending anonymous analytics information
[17:21:26]: Learn more at https://docs.fastlane.tools/#metrics
[17:21:26]: No personal or sensitive data is sent.
[17:21:26]: You can disable this by adding `opt_out_usage` at the top of your Fastfile
[17:21:26]: Error setting value 'action.xccovreport' for option 'xccov_file_direct_path'
[!] xccoverage/xccovreport/xcresult file does not exist
+ zip -r /Users/vagrant/deploy/xcov_report.zip 'reports/*'
	zip warning: name not matched: reports/*
zip error: Nothing to do! (try: zip -r /Users/vagrant/deploy/xcov_report.zip . -i reports/*)

from steps-xcode-test.

Brew install xcparse and Extract xccovreport were both successful. I'll try your edit above.

from steps-xcode-test.

I tried using the command directly on my local machine before setting up bitrise to do it:

bundle exec xcov -w CRM.xcworkspace -s CRM -f /Users/bdaniels/Library/Developer/Xcode/DerivedData/CRM-eadpxzelozpjpsbwotctvgpnhvoi/Logs/Test/Test-CRM-2019.10.25_10-42-06--0700.xcresult -o reports

and I got this error, which is the same error I got before:

Opening .xccoverage file at path: /var/folders/ry/zb2fv35j261fgt86bjndkl5m0000gp/T/d20191025-68799-1rwr3nw/action_0.xccovreport
Parsing .xccoverage file...
File successfully parsed
[13:04:08]: Exit status:

Looking for related GitHub issues on fastlane/fastlane...

Found no similar issues. To create a new issue, please visit:
https://github.com/fastlane/fastlane/issues/new
Run `fastlane env` to append the fastlane environment to your issue
bundler: failed to load command: xcov (/Users/bdaniels/.rbenv/versions/2.6.3/bin/xcov)
RuntimeError: [!] Error creating your coverage report - see the log above
  /Users/bdaniels/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/xcov-1.6.0/lib/xcov/error_handler.rb:20:in `handle_error'
...

from steps-xcode-test.

rather than having to use xcparse to get it for you

Are you saying xcparse is not necessary anymore?

from steps-xcode-test.

The issue could be because I'm using Catalina locally, however on Bitrise they don't offer Catalina yet and I'm still seeing an issue.

fastlane-community/xcov#159

from steps-xcode-test.

I tried using the command directly on my local machine before setting up bitrise to do it:

bundle exec xcov -w CRM.xcworkspace -s CRM -f /Users/bdaniels/Library/Developer/Xcode/DerivedData/CRM-eadpxzelozpjpsbwotctvgpnhvoi/Logs/Test/Test-CRM-2019.10.25_10-42-06--0700.xcresult -o reports

and I got this error, which is the same error I got before:

Opening .xccoverage file at path: /var/folders/ry/zb2fv35j261fgt86bjndkl5m0000gp/T/d20191025-68799-1rwr3nw/action_0.xccovreport
Parsing .xccoverage file...
File successfully parsed
[13:04:08]: Exit status:

Looking for related GitHub issues on fastlane/fastlane...

Found no similar issues. To create a new issue, please visit:
https://github.com/fastlane/fastlane/issues/new
Run `fastlane env` to append the fastlane environment to your issue
bundler: failed to load command: xcov (/Users/bdaniels/.rbenv/versions/2.6.3/bin/xcov)
RuntimeError: [!] Error creating your coverage report - see the log above
  /Users/bdaniels/.rbenv/versions/2.6.3/lib/ruby/gems/2.6.0/gems/xcov-1.6.0/lib/xcov/error_handler.rb:20:in `handle_error'
...

Yeah with xcov 1.6, you don't even need xcparse if you pass the xcresult directly. xcparse was being used as a crutch when xcov could only take the xccovreport directly. The error you got here seems like an issue with xcov separate from not finding the file now (and you are using xcov 1.6 so not an issue of an old version). You can see in the log it got to parsing an xccovreport supposedly extracted out. If you could get the full stacktrace of the RuntimeError, I'd open a new ticket over in the xcov project with that.

EDIT: @brandtdaniels Your other log before this one was previously failing due to the xccovreport being missing.

[17:21:26]: Error setting value 'action.xccovreport' for option 'xccov_file_direct_path'
[!] xccoverage/xccovreport/xcresult file does not exist

If when you did those you were getting xcparse to successfully try code coverage extraction, that sounds like maybe code coverage wasn't enabled in the test plan & xcparse saw there was no code coverage report and parsed nothing successfully (we don't error if there is no code coverage in the xcresult, we just don't create a xccovreport file then). That would assume though there's a bug in xcov where they create an xccovreport file even though the xcresult doesn't have one and then crashed when they find the file is empty

Opening .xccoverage file at path: /var/folders/ry/zb2fv35j261fgt86bjndkl5m0000gp/T/d20191025-68799-1rwr3nw/action_0.xccovreport
Parsing .xccoverage file...

Feel free to hit me up on email (it's on my profile) if you want to move debugging this off the public Bitrise issue tracker.

from steps-xcode-test.

Hello there, I'm a bot. On behalf of the community I thank you for opening this issue.

To help our human contributors focus on the most relevant reports, I check up on old issues to see if they're still relevant.
This issue has had no activity for 90 days, so I marked it as stale.

The community would appreciate if you could check if the issue still persists. If it isn't, please close it.
If the issue persists, and you'd like to remove the stale label, you simply need to leave a comment. Your comment can be as simple as "still important to me".

If no comment left within 21 days, this issue will be closed.

from steps-xcode-test.