Locating DVC Report about cml HOT 7 CLOSED

It's a very nice idea. @shcheklein @dmpetrov ?

like it! the only minor suggestion is to start with DVC: DVC: generated report or something like this. The reason is that a lot of people prefer all commits starting from lower case, a lot of folks with upper case - it will be painful for them to see our to breaking the rule :)

from cml.

Quick followup: I found the reports once I made a PR! I think we may have to make it very clear in the docs that the report is made when a PR occurs, not automatically whenever a GitHub action runs (which is what I expected).

Also, how can we generate a report comparing commits on the same branch? I might change some hyperparameters of my ML model on a single branch, rather than making a separate branch and PR for each experiment with a different set of hyperparameters. What makes sense for this case?

from cml.

Quick followup: I found the reports once I made a PR!

You should see also your reports in your commits list

Another possibility is the green arrow in the project

Unnamed Actions workflows are reports (are unnamed since the check is not trigger by a run)

and of course the PR widget as you mention.

Also, how can we generate a report comparing commits on the same branch?

Glad that you make that question. We have been working on that direction, however for this release we setup a baseline to be compared with current experiment. That baseline is always comparing with your origin/master or the commit, ref that you setup through baseline parameter. Aside of that you will see the last 5 experiments in your branch direct access.

from cml.

Cool, thanks for pointing these out! One note on the Commits view:

Only the commits with the message "dvc repro [ci skip]" generate a DVC report. As a person new to CI/CD, this was a bit confusing. Would it make sense to save the commit message as "Generate DVC report", or otherwise tag these commits somehow so I know to look here for the report?

from cml.

Cool, thanks for pointing these out!

Thanks to you to point this to improve the docs!

Only the commits with the message "dvc repro [ci skip]" generate a DVC report.

every commit generates a DVC Report, if repro does not have reason to run DVC-cml still generates a report (if you click the green arrow you will see two reports).

Would it make sense to save the commit message as "Generate DVC report",

It's a very nice idea. @shcheklein @dmpetrov ?

from cml.

It's a very nice idea. @shcheklein @dmpetrov ?

like it! the only minor suggestion is to start with DVC

Good idea! I'd not use DVC: prefix, I'd use DVC-CML: instead.

from cml.

Now reports can be found as Github Checks or Github/Gitlab commit comments that appears in the PR/MR workflow or commit overview page.

from cml.