Migration to `ctest` about cp2k HOT 4 OPEN

Thanks for the suggestion. I haven't really looked at ctest yet. As we're migrating to CMake we should presumably also integrate with ctest.

That being said, our do_regtests script has been refined for over 20 years and contains many bells and whistles that we have come to rely on. So, I don't think we can retire it anytime soon.

from cp2k.

That being said, our do_regtests script has been refined for over 20 years and contains many bells and whistles that we have come to rely on. So, I don't think we can retire it anytime soon.

I don't suggest to completely retire it, just to gradually remove some functionalities out of it, as things are being covered by ctest, particularly the scheduling. I think it's still a perfectly valid choice to have do_regtests to wrap the cp2k execution and tests, but I think it should be considered more carefully on what is included in the python script beyond the basic functionality of performing regression tests (re: #2996)

from cp2k.

PS: Feel free to check my template about a design for the ctest testsuite. It includes unit testing, regression testing, packaging tests, and example testing. If you want to discuss it in more details, let me know.

from cp2k.

Just saw that CP2K_ENABLE_REGTESTS makes it write files in the source folder. That is really bad and error prone in multi-build environments like when packaging and trying to build for multiple MPI environments. I understand that it was done to have minimal effect on the do_regtest.py, but still that creates dangerous setups for the end-user in order to accommodate the limitations of the test scripts.

from cp2k.