Test msrv against test about cargo-msrv HOT 4 CLOSED

Hi @Skallwar, that's indeed possible 😄. You can include the tests by running cargo msrv with the following arguments: cargo msrv -- cargo check --tests --all, or if you also want to execute the test cases cargo msrv -- cargo test --all. Since the test cases are generally not consumed by library consumers, these are not included by default.

In fact, any program runnable through rustup run can be used, and specified as the last argument, after the two dashes.

from cargo-msrv.

I won't claim to know whether there is a consensus on a best practice 🙂. In the projects of others I usually see tests being evaluated on both stable and the MSRV. I think that this configuration is used for at least two reasons:

• executing tests on MSRV as well (as stable) may have a small extra chance to catch out bugs caused by a toolchain (such as inadvertently introduced incompatibilities; such bugs are difficult to spot)
• it's easy to configure testing for both since usually you can simply add another toolchain version by including it in your build matrix
• if you have the build capacity, the stable and MSRV instances can run in parallel, not costing extra build time

There are also disadvantages to testing on additional platforms (e.g. MSRV) such as:

• if you have limited resources (e.g. for non public projects), CI minutes can become quite expensive
• if you have a smaller amount of parallel builds, or simply have a large test suite, the run time of the test suite may take a significant amount of time (which may also block other people from running their PR's on the CI workers)

For my own projects, I usually:

• run the tests on at least the latest backwards compatible stable
• run an MSRV instance with cargo check --all (i.e. building without linking) which has the aim of verifying consumers of a library can safely compile it as part of their project, within their own MSRV.

The test instances help me find defects within my code and configuration; the MSRV check tries to ensure a crate is able to run on a specific language feature set.

For me, this is usually the right balance between practicality and stability on this matter.

I'm however always interested in learning new developments in this area :).

from cargo-msrv.

I'm wondering if my CI should only run the test with a stable version and try to build with both stable and MSRV. I don't what is the best practice for this

from cargo-msrv.

Thanks a lot for taking the time to explain everything

from cargo-msrv.