boostorg / test Goto Github PK

View Code? Open in Web Editor NEW

177.0 19.0 140.0 11.21 MB

The reference C++ unit testing framework (TDD, xUnit, C++03/11/14/17)

Home Page: http://boost.org/libs/test

License: Boost Software License 1.0

C++ 99.03% HTML 0.02% CMake 0.90% Shell 0.05%

c-plus-plus boost unit-testing

test's Introduction

What is Boost.Test?

Boost.Test is a C++11/14/17 unit testing library, available on a wide range of platforms and compilers.

The library is part of Boost. The latest release of the library is available from the boost web site.

Full instructions for use of this library can be accessed from http://www.boost.org/doc/libs/release/libs/test/

Key features

Easy to get started with:

download and deflate the latest boost archive

create a test module with this (header version):

#define BOOST_TEST_MODULE your_test_module
#include <boost/test/included/unit_test.hpp>

Write your first test case:

BOOST_AUTO_TEST_CASE( your_test_case ) {
    std::vector<int> a{1, 2};
    std::vector<int> b{1, 2};
    BOOST_TEST( a == b );
}

build and run
done

powerful and unique test assertion macro BOOST_TEST, that understands floating points, collections, strings... and uses appropriate comparison paradigm
self-registering test cases, organize cases in test suites, apply fixtures on test cases, suites or globally
provide assertion context for advanced diagnostic on failure
powerful and extensible dataset tests
add decoration to test cases and suites for advanced description, group/label, and dependencies
powerful command line options and test case filters
extensible logging, XML and JUNIT outputs for third-party tools (eg. cont. integration)
various usage (shared/static library/header only) for faster integration and/or compilation/build cycles, smaller binaries

Copyright and license

Distributed under the Boost Software License, Version 1.0.
(Get a copy at www.boost.org/LICENSE_1_0.txt)

Contribute

Please read this document to get started.

Build Status

Boost.Test uses mostly the facility provided by our wonderful Boost testers (column Tests below).

Branch	Deps	Docs	Tests	Github Actions
`master`
`develop`

test's People

Contributors

Stargazers

Watchers

Forkers

microsoftarchive gjasny sanderbog timblechmann awulkiew marcelraad mkurdej ja11sop danieljames ming86 verisign jurko-gospodnetic gunnypong maxmus ricardh01 akrzemi1 kennysham1 akumta bjoe lanurmi jessicah libellus-biz limbolily winder aelkhour benwiederhake jwakely shital17 mauve leungs021185 nihildeb mfahd crossbuild mahmudandme apolukhin megatherium yueduiren kusuqettah-corp zerotypos-found gatlex mescogt rick68 stac47 mmr091 scottrigby faithandbrave madhurchadha houndie anydeploy rajjustrocks daptech27 autodesk-forks kojoley leeyifei danielae xltools oktal romain-geissler-1a dcbgh aargor norbertwenzel skozacik namezero111111 hia3 jeking3 bazelboost avikivity myestro shadowice jonnahtan lastique jschueller real-serious-games airtimemedia thughes cnsuhao qianqian121 toonetown mike-devel liangzhou9527 tobias-loew buckaroo-pm maksymilian-palka rainerdeyke ccf19881030 kedarbhat alalawi87 avcpp hazelnusse ecatmur kernigh citykspicychicken iamrameshkumar danielmevi giomasce zhaohui0923 valiko-ua ppmfloss f3rm4rf3r pkl97

test's Issues

Unable to compile boost/test for C++17

clang-linux.compile.c++.without-pth bin.v2/linux.static.x64/boost/bin.v2/libs/test/build/clang-linux-6.0.0/release/link-static/threading-multi/debug.o
In file included from libs/test/src/debug.cpp:16:
In file included from ./boost/test/impl/debug.ipp:53:
./boost/test/utils/algorithm.hpp:118:42: error: no member named 'bind1st' in namespace 'std'
        if( std::find_if( first2, last2, BOOST_TEST_BIND1ST( pred, *first1 ) ) == last2 )
                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./boost/test/utils/algorithm.hpp:25:38: note: expanded from macro 'BOOST_TEST_BIND1ST'
#define BOOST_TEST_BIND1ST(F,A) std::bind1st( (F), (A) )

boost_check_equal-str-test failed on llvm

Test log:
../libs/test/test/writing-test-ts/boost_check_equal-str-test.cpp(55): error: in "check_string_compare": check (void*)buf_ptr_cch != (void*)buf_array_cch has failed [0x800d3fd7 == 0x800d3fd7]

Test code:
char const* buf_ptr_cch = "abc";
char const buf_array_cch[] = "abc";
BOOST_TEST((void*)buf_ptr_cch != (void*)buf_array_cch);

Potential solution:
Add “clang:off” into boost.test-self-test() from libs/test/test/Jamfile.v2 to turn off the optimization for llvm. Verified the test can pass with this solution.

suppress -Wformat-overflow when optimization is enabled on GCC 8.2.0

Hi,

When I compile the following code with g++ v8.2.0 on linux (Ubuntu 16.04),

#define BOOST_TEST_MODULE test_min
#include <boost/test/included/unit_test.hpp>

BOOST_AUTO_TEST_CASE(minimal)
{
    const double a = 1.5, b = 2.5, c = 4.0;
    BOOST_TEST(c == a + b, boost::test_tools::tolerance(1e-8));
}

I get this warning.

$ g++-8 test.cpp -Wformat-overflow=1 -O2 -I./boost_1_68_0
In file included from ./boost_1_68_0/boost/test/impl/compiler_log_formatter.ipp:29,
                 from ./boost_1_68_0/boost/test/included/unit_test.hpp:18,
                 from test.cpp:2:
./boost_1_68_0/boost/test/utils/setcolor.hpp: In constructor ‘boost::unit_test::utils::scope_setcolor::scope_setcolor(std::ostream&, boost::unit_test::utils::term_attr::_, boost::unit_test::utils::term_color::_, boost::unit_test::utils::term_color::_)’:
./boost_1_68_0/boost/test/utils/setcolor.hpp:84:38: warning: ‘%d’ directive writing between 1 and 11 bytes into a region of size between 0 and 9 [-Wformat-overflow=]
         m_command_size = std::sprintf( m_control_command, "%c[%d;%d;%dm", 0x1B, attr, fg + 30, bg + 40 );
                          ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./boost_1_68_0/boost/test/utils/setcolor.hpp:84:38: note: directive argument in the range [-2147483618, 2147483647]
./boost_1_68_0/boost/test/utils/setcolor.hpp:84:38: note: directive argument in the range [-2147483608, 2147483647]
./boost_1_68_0/boost/test/utils/setcolor.hpp:84:38: note: ‘sprintf’ output between 9 and 39 bytes into a destination of size 13

And the version of g++ is this.

$ g++-8 --version
g++ (GCC) 8.2.0
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Also, I found that it happens only when I turn optimization on. With -O0 (or without -O flags), I found no warning with -Wformat-overflow=1. You can check it here.

I know in reality it does not matter because the types of arguments are actually enum types and the range of the defined value is in [0, 9]. Even though, I would like to suppress it.

As the message said, the reason for this warning is that the buffer is too short. To supress the warning, basically, we need to increase the buffer size, or use std::snprintf instead. However std::snprintf doesn't exist in C++98, so we cannot use it. I think expanding the buffer is the best way if the size of the class setcolor does not matter.

I have one more workaround that does not need to expand the buffer. Assuming ASCII, we can convert an integral value to ASCII code and put it exactly as a byte. To make the argument as just one character, we can write 3 and 4 directly in the format string instead of adding 30 and 40 to the argument.

This workaround would look like this. I confirmed that it suppresses the warning.

m_command_size = std::sprintf( m_control_command, "%c[%c;3%c;4%cm", 0x1B,
    static_cast<char>(attr + 48), static_cast<char>(fg + 48), static_cast<char>(bg + 48));

However, since POSIX does not define the character encoding, it is not a portable code.

What do you think about this problem?

Thanks.

Provide more flexible ways for outputting the tested types

According to the docs, if I want a custom output from tests for my type, I either have to provide an ADL-visible operator<< or ADL-visible function boost_test_print_type. This is not satisfactory in a number of cases.

1. Sometimes I operate on pointers or pairs:

struct Item
{
  int uid;
  string name;
  string title;
};

using ItemPtr = std::shared_ptr<Item>;
using Count = std::pair<Item, int>;

And I would like to serialize ItemPtr p by p->uid.

I cannot do this because I would have to add a declaration to namespace std which is an undefined behavior.

2. Sometimes I want to display the same object in two different ways in two different test cases.

In a different test case I may want to display ItemPtr p as

 "{"s + p->name + "," + p->title + "}"

Issue 1 could be addressed in two ways.

The library provides a class template boost::test::print_type<T>::print(T const&, std::ostream&) which by default goes to boost_test_print_type but I can provide a specialization for my types.
You provide a function ADL-based interface, but one of the arguments is from boost::test namespace:

namespace boost { namespace test {

  struct tag {};

  template <typename T>
  void print_type(T const&, std::ostream&, tag); // default impl

}}

This way, I can always add overload into namespace boost::test:

namespace boost { namespace test {

  void print_type(std::shared_ptr<int> const&, std::ostream&, tag); // overload

}}

Tags could also solve the second issue, if they were somehow associated with test cases of test case fixtures. Or if rather than tags one would use fixture or test case namesnames:

namespace boost { namespace test {

  template <typename TT>
  struct tag {};

  template <typename T, typename TT>
  void print_type(T const&, std::ostream&, tag<TT>); // default impl

}}

namespace boost { namespace test {

  // overloads:
  void print_type(std::shared_ptr<int> const&, std::ostream&, tag<fixture_1>); 
  void print_type(std::shared_ptr<int> const&, std::ostream&, tag<fixture_2>); 

}}

junit report: test error is also reported as failure

Consider this snippet:

#define BOOST_TEST_MODULE Test1
#include <boost/test/included/unit_test.hpp>

BOOST_AUTO_TEST_CASE(testexception)
{
 throw 42;
}

Compiling and running above sample yields:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite tests="0" skipped="0" errors="1" failures="1" id="0" name="Test1" time="0.000137">
<testcase assertions="1" name="testexception" time="0.000137">

Now, this is not so great. JUnit would report this as:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite tests="0" skipped="0" errors="1" failures="0" id="0" name="Test1" time="0.000137">
<testcase assertions="1" name="testexception" time="0.000137">

Because according to JUnit a test either succeeds or fails (one or more assertion failures) or produces an error (unexpected failure). Boost on the other hand thinks that an error is also a failure.

Reproduced with Boost 1.66.0

Compile time error with MinGW and boost/test/minimal.hpp

Attempt to build Boost.Variant tests with MinGW end with the following error:

In file included from ..\..\../boost/test/minimal.hpp:53:0,
                 from variant_multivisit_test.cpp:18:
..\..\../boost/test/impl/execution_monitor.ipp: In function 'void boost::detail::report_error(boost::execution_exception::error_code, const boost::exception*, const char*, char**)':
..\..\../boost/test/impl/execution_monitor.ipp:221:84: error: 'vsnprintf' was not declared in this scope
 #  define BOOST_TEST_VSNPRINTF( a1, a2, a3, a4 ) vsnprintf( (a1), (a2), (a3), (a4) )
                                                                                    ^
..\..\../boost/test/impl/execution_monitor.ipp:246:5: note: in expansion of macro 'BOOST_TEST_VSNPRINTF'
     BOOST_TEST_VSNPRINTF( buf, sizeof(buf)-1, format, *args );
     ^

Full log available at https://ci.appveyor.com/project/apolukhin/variant/build/job/hq2ob6ipas8g4qrc

UBSAN issue with --log_level=all

Was compiling Boost.Thread for UBSAN and ran into a problem with a test. I wasn't sure which test it was so I tried running the test with --log_level=all, and I saw a different error in Boost.Test.

This is reproducible in the Boost Docker Development Environment. To reproduce first we build the test for ubsan and see an issue for Boost.Thread (not this issue - keep reading - this issue is after):

jking@ubuntu:~/boost/libs/thread/test$ bdde
docker run -v /home/jking/boost:/boost:rw -v /home/jking/bdde:/bdde:ro --workdir /boost/libs/thread/test -it jeking3/bdde:linux /bin/bash
boost@a84a7e1cdcec:/boost/libs/thread/test$ ubsan test_thread_exit
UBSAN_OPTIONS=print_stacktrace=1 b2 define=BOOST_NO_STRESS_TEST=1 cxxflags=-fno-omit-frame-pointer cxxflags=-fsanitize=undefined cxxflags=-fno-sanitize-recover=undefined linkflags=-fsanitize=undefined linkflags=-fno-sanitize-recover=undefined linkflags=-fuse-ld=gold variant=debug toolset=gcc-8 test_thread_exit
Performing configuration checks

    - default address-model    : 64-bit (cached)
    - default architecture     : x86 (cached)
    - symlinks supported       : yes (cached)
    - BOOST_COMP_GNUC >= 4.3.0 : yes (cached)
    - lockfree boost::atomic_flag : yes (cached)
...patience...
...patience...
...found 3380 targets...
...updating 2 targets...
testing.capture-output ../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit.run
====== BEGIN OUTPUT ======
../../../libs/thread/src/pthread/thread.cpp:103:58: runtime error: member call on address 0x7f44d0000b20 which does not point to an object of type 'thread_exit_function_base'
0x7f44d0000b20: note: object is of type 'boost::detail::thread_exit_function<void (*)()>'
 00 00 00 00  b0 4f 0a ce 15 56 00 00  cb 07 07 ce 15 56 00 00  00 00 00 00 00 00 00 00  25 00 00 00
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'boost::detail::thread_exit_function<void (*)()>'

EXIT STATUS: 1
====== END OUTPUT ======

    LD_LIBRARY_PATH="/boost/bin.v2/libs/chrono/build/gcc-8/debug/threading-multi/visibility-hidden:/boost/bin.v2/libs/system/build/gcc-8/debug/threading-multi/visibility-hidden:/boost/bin.v2/libs/test/build/gcc-8/debug/threading-multi/visibility-hidden:/boost/bin.v2/libs/thread/build/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden:/boost/bin.v2/libs/timer/build/gcc-8/debug/threading-multi/visibility-hidden:/usr/bin:/usr/lib:/usr/lib32:/usr/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH

    status=0
    if test $status -ne 0 ; then
        echo Skipping test execution due to testing.execute=off
        exit 0
    fi
     "../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit"   > "../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit.output" 2>&1 < /dev/null
    status=$?
    echo >> "../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit.output"
    echo EXIT STATUS: $status >> "../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit.output"
    if test $status -eq 0 ; then
        cp "../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit.output" "../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit.run"
    fi
    verbose=0
    if test $status -ne 0 ; then
        verbose=1
    fi
    if test $verbose -eq 1 ; then
        echo ====== BEGIN OUTPUT ======
        cat "../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit.output"
        echo ====== END OUTPUT ======
    fi
    exit $status

...failed testing.capture-output ../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit.run...
...failed updating 1 target...
...skipped 1 target...

Now we run the test with --log_level=all and see this issue:

boost@a84a7e1cdcec:/boost/libs/thread/test$ UBSAN_OPTIONS=print_stacktrace=1 ../../../bin.v2/libs/thread/test/test_thread_exit.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_thread_exit --log_level=all
Running 2 test cases...
Entering test module "Boost.Threads: thread exit test suite"
test_thread_exit.cpp(32): Entering test case "test_thread_exit_func_runs_when_thread_exits"
../../../boost/test/impl/unit_test_log.ipp:451:134: runtime error: member call on address 0x7f423a21adf0 which does not point to an object of type 'lazy_ostream'
0x7f423a21adf0: note: object is of type 'boost::unit_test::lazy_ostream_impl<boost::unit_test::lazy_ostream, boost::unit_test::basic_cstring<char const>, boost::unit_test::basic_cstring<char const> const&>'
 00 00 00 00  28 30 47 57 9f 55 00 00  00 aa 46 57 9f 55 00 00  40 02 49 57 9f 55 00 00  b0 ad 21 3a
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'boost::unit_test::lazy_ostream_impl<boost::unit_test::lazy_ostream, boost::unit_test::basic_cstring<char const>, boost::unit_test::basic_cstring<char const> const&>'
    #0 0x7f423c5fd4b3 in boost::unit_test::unit_test_log_t::operator<<(boost::unit_test::lazy_ostream const&) ../../../boost/test/impl/unit_test_log.ipp:451
    #1 0x7f423c5ca8cc in void boost::test_tools::tt_detail::format_report<boost::unit_test::unit_test_log_t>(boost::unit_test::unit_test_log_t&, boost::test_tools::assertion_result const&, boost::unit_test::lazy_ostream const&, boost::test_tools::tt_detail::tool_level, boost::test_tools::tt_detail::check_type, unsigned long, __va_list_tag*, char const*, char const*) ../../../boost/test/impl/test_tools.ipp:144
    #2 0x7f423c5bf9fa in boost::test_tools::tt_detail::report_assertion(boost::test_tools::assertion_result const&, boost::unit_test::lazy_ostream const&, boost::unit_test::basic_cstring<char const>, unsigned long, boost::test_tools::tt_detail::tool_level, boost::test_tools::tt_detail::check_type, unsigned long, ...) ../../../boost/test/impl/test_tools.ipp:355
    #3 0x559f5743eaea in tf1() /boost/libs/thread/test/test_thread_exit.cpp:29
    #4 0x559f57467b5a in boost::detail::thread_data<void (*)()>::run() ../../../boost/thread/detail/thread.hpp:117
    #5 0x7f423ca6f307 in thread_proxy ../../../libs/thread/src/pthread/thread.cpp:177
    #6 0x7f423adbe6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
    #7 0x7f423aae788e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x12188e)

Test from boost libraries failed to run after boost submodule test updated from b56dc6(master) to f24f8f

Environment:
VS 2017 + Windows Server 2016

Issue description:
We build and run test for boost. And we found one test failed to run after boost submodule test updated from b56dc6(master) to f24f8f [(boostorg/boost@68a1e99)] . Details is as below. Could you please take a look?

Reproduce steps:

git clone -c core.autocrlf=true --recursive https://github.com/boostorg/boost.git D:\Boost\src
open a VS 2017 x86 command prompt and browse to D:\Boost\src
.\bootstrap
.\b2 headers variant=release --build-dir=..\out\x86rel address-model=32
.\b2 variant=release --build-dir=..\out\x86rel address-model=32
.\b2 -j4 variant=release --build-dir=..\out\x86rel libs\test\test

log_x86_test_104.log

ErrorMessage:
====== BEGIN OUTPUT ======
Running 1 test case...

*** 1 failure is detected in the test suite "fake master"
libs/test/test/execution_monitor-ts/boost_exception-test.cpp(63): info: check error_string.find("tag_error_code") != std::string::npos has passed
libs/test/test/execution_monitor-ts/boost_exception-test.cpp(64): info: check error_string.find("= 123") != std::string::npos has passed
libs/test/test/execution_monitor-ts/boost_exception-test.cpp(65): info: check error_string.find("Dynamic exception type") != std::string::npos has passed
libs/test/test/execution_monitor-ts/boost_exception-test.cpp(66): error: in "test_logs": check error_string.find("boost::wrapexcept<") != std::string::npos has failed [4294967295 == 4294967295]
libs/test/test/execution_monitor-ts/boost_exception-test.cpp(67): info: check error_string.find("myexception>") != std::string::npos has passed
libs\test\test\execution_monitor-ts\boost_exception-test.cpp(43): Leaving test case "test_logs"; testing time: 2ms
Leaving test module "Master Test Suite"; testing time: 2ms

*** 1 failure is detected in the test module "Master Test Suite"

EXIT STATUS: 201
====== END OUTPUT ======

High arity data test cases limit set by boost::bind() is much too low.

When having models with a large set of input parameters, one cannot use BOOST_DATA_TEST_CASE(). e.g in my case I need 12 input parameters.

It may be reasonable to not support binding of more than 9 arguments. But not supporting test cases with ten input parameters is not. One could e.g offer something like BOOST_DATA_TUPLE_TEST_CASE() that stuffs all arguments into a tuple and supplies that to the user.

Unhandled exceptions while debugging with MS Visual Studio

I'm using some ancient version of boost.test (perhaps 1.58), however, by looking at the latest code I think latest version has the same issue. Boost.test captures c++ exceptions as well as SEH to be able to handle all kinds of errors that normally result in crashes. The problem with that, however, it makes debugging tests difficult. I have some test that somehow triggers an unhanded C++ in my library. Normally, unhandled exception gets handled by the visual studio debugger and it stops basically on the line where exception is thrown and it's clear and obvious where it came from. With boost.test: forget about it, no way to see anything.

So, how can I make boost.test not try to capture SEH or c++ exceptions while debugger is attached? For now I added code to wrap test_method calls in BOOST_FIXTURE_TEST_CASE to capture SEH myself and write a dmp (core dump) for later debugging, but I would definitely prefer that boost.test wouldn't try to capture unhandled c++ or SEH exceptions when debug_present. What can be done to have this behavior, does boost.test have it?

close_at_tolerance always returns false for comparisons of infinity

Consider this code:

#include <boost/test/tools/floating_point_comparison.hpp>
#include <iostream>
#include <limits>

namespace fpc = boost::math::fpc;
int main() {
	float a = std::numeric_limits<float>::infinity();
	float b = std::numeric_limits<float>::infinity();

	auto P = fpc::close_at_tolerance<float>(fpc::percent_tolerance<float>(0.01), fpc::FPC_WEAK);
	std::cout << P(a, b) << "\n";
}

It prints 0. You might say "well that makes sense because one infinity is not the same as another". Except that the IEEE floating point standard considers inf == inf to be true, which means that you have the strange situation where this passes:

BOOST_AUTO_TEST_CASE(foo)
) {
	float a = std::numeric_limits<float>::infinity();
	float b = a;
	BOOST_TEST(a == b);
}

But this does not:

BOOST_AUTO_TEST_CASE(foo,
	*utf::tolerance<float>(fpc::percent_tolerance<float>(0.01))
) {
	float a = std::numeric_limits<float>::infinity();
	float b = a;
	BOOST_TEST(a == b);
}

Also, I'm 90% sure the latter case did pass in the past (Boost 1.60 at least).

Testing the main() function

I'm trying to adopt a test-driven approach for my introductory programming class where our first topics are on variable declarations, using cout, simple conditions, and loops. Students don't know how to use functions yet, so I was wondering if it was possible to run a unit test on main.

I've seen a couple of discussions that suggest ways to grab data from cout (see accepted answer) that then allow testing, but the issue is that students' programs never really call functions. I don't know how to call the main function from the unit test, if that's even possible. Here's sample code of the types of programs students make to give some context.

#include <iostream>
using namespace std;

int main()
{
  int num1;
  int num2;
  cout << "Please enter the first number: ";
  cin >> num1;
  cout << "Please enter the second number: ";
  cin >> num2;
  cout << "The sum of " << num1 << " and " << num2 << " is " << num1 + num2 << endl;
  return 0;
}

UBSAN identified a problem at exit time by gcc-8 only

During a build of Boost.Rational with UBSAN and gcc-8 - this issue is reproducible in boostorg/rational#30.

testing.capture-output ../../bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test.run
====== BEGIN OUTPUT ======
Running tests for boost::rational<long>
Implementation issue: the minimal size for a rational
is twice the size of the underlying integer type.
Checking to see if space is being wasted.
	sizeof(long) == 8
	sizeof(boost::rational<long>) == 16
Implementation has minimal size
Running 83 test cases...
*** No errors detected
../../boost/test/impl/framework.ipp:839:34: runtime error: member call on address 0x000000b5caa0 which does not point to an object of type 'test_observer'
0x000000b5caa0: note: object has invalid vptr
 00 00 00 00  00 00 00 00 00 00 00 00  01 00 00 00 00 00 00 00  d8 cb b5 00 00 00 00 00  00 00 00 00
              ^~~~~~~~~~~~~~~~~~~~~~~
              invalid vptr
    #0 0x6656bb in boost::unit_test::framework::state::priority_order::operator()(boost::unit_test::test_observer*, boost::unit_test::test_observer*) const ../../boost/test/impl/framework.ipp:839
    #1 0x695d5d in std::_Rb_tree<boost::unit_test::test_observer*, boost::unit_test::test_observer*, std::_Identity<boost::unit_test::test_observer*>, boost::unit_test::framework::state::priority_order, std::allocator<boost::unit_test::test_observer*> >::_M_lower_bound(std::_Rb_tree_node<boost::unit_test::test_observer*>*, std::_Rb_tree_node_base*, boost::unit_test::test_observer* const&) (/home/travis/build/boostorg/boost-root/bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test+0x695d5d)
    #2 0x68db46 in std::_Rb_tree<boost::unit_test::test_observer*, boost::unit_test::test_observer*, std::_Identity<boost::unit_test::test_observer*>, boost::unit_test::framework::state::priority_order, std::allocator<boost::unit_test::test_observer*> >::equal_range(boost::unit_test::test_observer* const&) /usr/include/c++/8/bits/stl_tree.h:1966
    #3 0x67dea5 in std::_Rb_tree<boost::unit_test::test_observer*, boost::unit_test::test_observer*, std::_Identity<boost::unit_test::test_observer*>, boost::unit_test::framework::state::priority_order, std::allocator<boost::unit_test::test_observer*> >::erase(boost::unit_test::test_observer* const&) /usr/include/c++/8/bits/stl_tree.h:2516
    #4 0x66f666 in std::set<boost::unit_test::test_observer*, boost::unit_test::framework::state::priority_order, std::allocator<boost::unit_test::test_observer*> >::erase(boost::unit_test::test_observer* const&) /usr/include/c++/8/bits/stl_set.h:685
    #5 0x64662d in boost::unit_test::framework::deregister_observer(boost::unit_test::test_observer&) ../../boost/test/impl/framework.ipp:1343
    #6 0x6e3d95 in boost::unit_test::global_configuration::~global_configuration() ../../boost/test/impl/test_tree.ipp:556
    #7 0x4db2b4 in ~global_configuration_impl ../../boost/test/tree/global_fixture.hpp:68
    #8 0x7fe46ef1e1a8  (/lib/x86_64-linux-gnu/libc.so.6+0x3c1a8)
    #9 0x7fe46ef1e1f4 in exit (/lib/x86_64-linux-gnu/libc.so.6+0x3c1f4)
    #10 0x7fe46ef03f4b in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21f4b)
    #11 0x40a008  (/home/travis/build/boostorg/boost-root/bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test+0x40a008)
EXIT STATUS: 1
====== END OUTPUT ======
    LD_LIBRARY_PATH="/usr/bin:/usr/lib:/usr/lib32:/usr/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
    status=0
    if test $status -ne 0 ; then
        echo Skipping test execution due to testing.execute=off
        exit 0
    fi
     "../../bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test"   > "../../bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test.output" 2>&1 < /dev/null
    status=$?
    echo >> "../../bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test.output"
    echo EXIT STATUS: $status >> "../../bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test.output"
    if test $status -eq 0 ; then
        cp "../../bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test.output" "../../bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test.run"
    fi
    verbose=0
    if test $status -ne 0 ; then
        verbose=1
    fi
    if test $verbose -eq 1 ; then
        echo ====== BEGIN OUTPUT ======
        cat "../../bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test.output"
        echo ====== END OUTPUT ======
    fi
    exit $status
...failed testing.capture-output ../../bin.v2/libs/rational/test/rational_test.test/gcc-8/debug/cxxstd-03-iso/visibility-hidden/rational_test.run...

see: https://travis-ci.org/boostorg/rational/jobs/447297672

Conditionally skipping a test case of a test suite causes failure of the test run

When conditionally skipping a test case, which is a member of a test suite, the suite fails.
A skipped test should just increment the skip count but not fail the test run.

Reproducer

shell:

mkdir build && cd build && cmake .. && make
# The next test run succeeds: all test cases are ran
./disable_test_case
# The next test run fails: some test cases are skipped
./disable_test_case -- --disable

CMakeLists.txt:

cmake_minimum_required(VERSION 3.8)
project(boost_test_disable_suite)

find_package(Boost COMPONENTS unit_test_framework program_options)

add_executable(disable_test_case
    main.cpp
    )

target_link_libraries(disable_test_case
    PUBLIC
        Boost::unit_test_framework
        Boost::program_options
    )

target_compile_definitions(disable_test_case
    PUBLIC
        BOOST_ALL_DYN_LINK
    )

main.cpp:

#define BOOST_TEST_MODULE disable_test_suite
#include <boost/test/unit_test.hpp>
#include <boost/test/results_collector.hpp>

#include <boost/program_options.hpp>

#include <iostream>

namespace utf = boost::unit_test;
namespace tt = boost::test_tools;

static bool globalFixtureRan = false;
static bool enableTests = true;

class GlobalFixture {
public:
    GlobalFixture() {
        int argc = utf::framework::master_test_suite().argc;
        char** argv= utf::framework::master_test_suite().argv;
        parse_args(argc, argv);

        globalFixtureRan = true;
    }
private:
    void parse_args(int argc, char** argv) {
        namespace po = boost::program_options;
        po::options_description test_options("Test options");
        test_options.add_options()(
            "disable", "Disable some tests")(
            "help", "Show this help message");

        po::variables_map vm;
        try {
            po::store(po::parse_command_line(argc, argv, test_options), vm);
            po::notify(vm);
        } catch (po::error& ex) {
            std::cout << "Error parsing arguments: " << ex.what() << std::endl;
            std::cerr << test_options;
            _exit(EXIT_FAILURE);
        }
        if (vm.count("help")) {
            std::cout << test_options;
            _exit(EXIT_SUCCESS);
        }

        if (vm.count("disable")) {
            enableTests = false;
        }
    }
};

BOOST_TEST_GLOBAL_FIXTURE(GlobalFixture);

struct enable_test {
    tt::assertion_result operator()(utf::test_unit_id id) {
        tt::assertion_result ans(enableTests);
        if (!enableTests) {
            auto& master = utf::framework::master_test_suite();
            master.remove(id);
        }
        return enableTests;
    }
};

BOOST_AUTO_TEST_SUITE(SecondSuccessSuiteWithConditionalTestCase)
BOOST_AUTO_TEST_CASE(conditional_success,
                     * utf::precondition(enable_test{})) {
    BOOST_REQUIRE(true);
}
BOOST_AUTO_TEST_CASE(unconditional_success) {
    BOOST_REQUIRE(true);
}
BOOST_AUTO_TEST_SUITE_END()

Document required linker flags

When using BOOST_TEST_DYN_LINK, the linker flag -lboost_unit_test_framework is not sufficient to successfully compile. Apparently there are more libraries that must be linked. For instance, when linking only with -lboost_unit_test_framework, I get the error:

Undefined symbols for architecture x86_64:
  "boost::unit_test::unit_test_log_t::instance()", referenced from:
      ___cxx_global_var_init in lanczos_smoothing_test-05d504.o
  "boost::unit_test::decorator::collector_t::instance()", referenced from:
      ___cxx_global_var_init.3 in lanczos_smoothing_test-05d504.o
  "boost::unit_test::ut_detail::auto_test_unit_registrar::auto_test_unit_registrar(boost::unit_test::test_case*, boost::unit_test::decorator::collector_t&, unsigned long)", referenced from:
      ___cxx_global_var_init.3 in lanczos_smoothing_test-05d504.o
ld: symbol(s) not found for architecture x86_64
clang: fatal error: linker command failed with exit code 1 (use -v to see invocation)

Setting color_output=no does not disable the output of color format codes

I think --color_output=no should not just use the default color for everything, it should stop color format codes from being outputted completely. The current behaviour is very misleading to me.

test/include/boost/test/utils/setcolor.hpp

Line 304 in 35c9acf

is_color_output ? utils::attr : utils::term_attr::NORMAL, \

To understand why it is a problem let me describe my test setup:
I use ctest to run boost tests and instructed ctest to output everything to a file.
The file contains all color codes that boost writes, --color_output=yes/no only changes if its all the same color or not, the codes remain any way.

Support for Boost.MP11 and Boost.Hana type lists?

BOOST_AUTO_TEST_CASE_TEMPLATE uses Boost.MPL's machinery (in particular: boost::mpl::for_each) to automate tests cases over type lists. However, Boost.MPL is not compatible with the modern Boost.MP11 and Boost.Hana type lists, firing BOOST_MPL_ASSERT(( is_sequence<Sequence> )); when passing an mp11::mp_list or a hana::tuple instead of an mpl::vector.

Is it possible to swap out the Boost.MPL implementation for Boost.MP11? I think Boost.MP11 uses only pattern matching on variadic parameter packs to determine if something is a type list, so it should be backward compatible with existing code that passes mpl::vector as type lists, or even with Boost.Hana (or std::tuple) type lists. To accomodate C++98 users, using an MP11 implementation can be done conditionally on the compiler version.

Timeout effect on Windows

I am running a set of tests on Windows Visual Studio 2017, using boost 1.66 in debug mode

this is the result I get from a set of tests

Running 14 test cases...

0%   10   20   30   40   50   60   70   80   90   100%
|----|----|----|----|----|----|----|----|----|----|
***************************************************

*** Errors were detected in the test module "key_tests"; see standard output for details
Press any key to continue . . .

this is a subset of a much larger test suite

After digging down I finally figured out what was happening is that this test is a data test cases and has a timeout decorator and it was skipping the last few of the tests

BOOST_TEST_DECORATOR (*utf::timeout (5))
BOOST_DATA_TEST_CASE (...

if report_level is set to detailed I get some additional explanation

Test module "key_tests" has failed with:
  10 test cases out of 14 passed
  4 test cases out of 14 skipped
  120 assertions out of 120 passed

with some later indicators stating that the last few tests were skipped.
I was expecting the decorator to apply to the individual tests vs the entire data test loop as well, but that may be my mistaken assumption.

There are a couple issues with this. The first is that
the documentation
states the timeout decorator has no effect on windows, so this appears now wrong, we have the timeouts in place as the test suites also run on Linux quite frequently.

And second some failure messages in the standard output would be much appreciated as I had no idea what was going on when this first started showing up.

error: comparison of integers of different signs

I got 3 errors while compiling https://github.com/wesnoth/wesnoth unit_tests
Log of errors:
http://ix.io/FhW

I got them when I doesn't have isystem ./Headers flag set in C compiler flags.

Compare collections of floating point values with tolerance

Currently you can use either:

BOOST_TEST_REQUIRE( output == outputDesired, boost::test_tools::per_element() );
BOOST_TEST_REQUIRE( output == outputDesired, boost::test_tools::tolerance( 1e-12f ) );

But you cannot use both.

What is the preferred way to compare collections (e.g. vectors) of floating point values?

This is a common use case for testing numeric algorithms: Check results of optimized version vs. naive implementation.

shared-library-test, shared-library-custom-init-test, shared-library-custom-main-test failed to build when running with “static-only=on”

Build log:
clang-vxworks.link ../bin.v2/libs/test/test/shared-library-test.test/clang-vxworks/debug/cross-compile-vxworks/static-only-on/shared-library-test.vxe
../crt0.o: In function _start': crtrtp.c:(.text+0x58): undefined reference to main'
../bin.v2/libs/test/test/shared-library-test.test/clang-vxworks/debug/cross-compile-vxworks/static-only-on/usage-variants-ts/shared-library-test.o: In function init_unit_test_suite(int, char**)': ../boost/test/unit_test_suite.hpp:383: undefined reference to boost::unit_test::framework::master_test_suite()'
../bin.v2/libs/test/test/shared-library-test.test/clang-vxworks/debug/cross-compile-vxworks/static-only-on/usage-variants-ts/shared-library-test.o: In function test_invoker()': ../libs/test/test/usage-variants-ts/shared-library-test.cpp:19: undefined reference to boost::unit_test::unit_test_log_t::set_checkpoint(boost::unit_test::basic_cstring, unsigned int, boost::unit_test::basic_cstring)'

clang-vxworks.compile.c++ ../bin.v2/libs/test/test/shared-library-custom-init-test.test/clang-vxworks/debug/cross-compile-vxworks/static-only-on/usage-variants-ts/shared-library-custom-init-test.o
../libs/test/test/usage-variants-ts/shared-library-custom-init-test.cpp:36:44: error: cannot initialize a parameter of type 'boost::unit_test::init_unit_test_func' (aka 'boost::unit_test::test_suite ()(int, char **)') with an rvalue of type 'bool (*)()': different number of parameters (2 vs 0)
return boost::unit_test::unit_test_main( &init_unit_test, argc, argv );
^~~~~~~~~~~~~~~

clang-vxworks.compile.c++ ../bin.v2/libs/test/test/shared-library-custom-main-test.test/clang-vxworks/debug/cross-compile-vxworks/static-only-on/usage-variants-ts/shared-library-custom-main-test.o
../libs/test/test/usage-variants-ts/shared-library-custom-main-test.cpp:30:45: error: use of undeclared identifier 'init_unit_test'
return boost::unit_test::unit_test_main( &init_unit_test, argc, argv );

Potential solution:
Update libs/test/test/Jamfile.v2 the following lines, to add ": : : : : shared" at the end of the lines:
[ boost.test-self-test run : usage-variants-ts : shared-library-test : boost_unit_test_framework/shared : : : : : shared ]
[ boost.test-self-test run : usage-variants-ts : shared-library-custom-init-test : boost_unit_test_framework/shared : : : : : shared ]
[ boost.test-self-test run : usage-variants-ts : shared-library-custom-main-test : boost_unit_test_framework/shared : : : : : shared ]

So when boost is running with “static-only=on”, these tests can be disabled by adding "on,shared:no" in our project requirements.

Test name should handle const-volatile specifiers

See comment here

BOOST_CHECK_CLOSE(0, smallnumber) fails

Hello
With boost 1.60
BOOST_CHECK_CLOSE(-0, -4.37113883e-08, 1)
fails with :
difference{3.40282e+38} between -0 and -4.37113883e-08 exceeds 1%
Note the difference is super big.
Should nt it pass ?
Tks

test_tools-test failed on some archs

Test log:
../libs/test/test/writing-test-ts/test_tools-test.cpp(363): error: in "test_BOOST_CHECK_EQUAL": check ots().match_pattern() has failed. Mismatch at position 721 (line 8, column 69): 'e' != 'f' :
... ailed [0xfe != 0xfd] ...
... ailed [0xfffffffe != ...

Test code:
TEST_CASE( test_BOOST_CHECK_EQUAL )
{
…
char ch1 = -2;
char ch2 = -3;
BOOST_CHECK_EQUAL(ch1, ch2);
}

Potential solution:
This is because char is unsigned on some archs.
The test code can be fixed by changing the type of the variables ‘ch1’ & ‘ch2’, from ‘char’ to ‘signed char’.
Then adding the following code into test_tools-test.cpp:
namespace boost{ namespace test_tools{ namespace tt_detail{
template<>
struct print_log_value {
void operator()( std::ostream& ostr, signed char t )
{
ostr << std::hex
#if BOOST_TEST_USE_STD_LOCALE
<< std::showbase
#else
<< "0x"
#endif
<< static_cast(t);
}
};
}}}

Verified the test can pass with the solution.

Total number of tests is greater when running with --run_test=* option

I have a Boost.Test project with 8772 test cases but today I spent a few hours trying to find ONE test which is not run on our build server because when I run it locally with --run_test=* the number of tests is greater by 1.

Why?

Can't directly compare optional<int> to nullopt under GCC

The following code compiles under Clang but not under GCC:

#include <boost/test/unit_test.hpp>
#include <optional>

BOOST_TEST_DONT_PRINT_LOG_VALUE( ::std::nullopt_t     )
BOOST_TEST_DONT_PRINT_LOG_VALUE( ::std::optional<int> )

BOOST_AUTO_TEST_CASE(y) {
    BOOST_TEST( ::std::make_optional( 5 ) == ::std::nullopt );
}

I suspect that there's an issue with GCC's std::is_constructible needlessly explicitly instantiating boost::test_tools::assertion::expression_base's operator bool() and I've raised 87093 about that.

However, (a) the GCC devs may well reject that issue and (b) I suspect it's workaround-able either way and it'd be really nice to be able to express this directly. Please can you look into this?

Thanks for all your work on this library.

boost_test_print_type is not considered when printing out values with non-explicit operator bools

struct s
{
	operator bool() const { return false; }
};
std::ostream &boost_test_print_type(std::ostream &o, const s &) { return o << "s"; }

struct s2
{
	operator bool() const { return false; }
};
std::ostream &operator<<(std::ostream &o, const s2 &) { return o << "s2"; }

BOOST_AUTO_TEST_CASE(f)
{
	BOOST_TEST(s{});
	BOOST_TEST(s2{});
}

When printing out the failures in this test, the second check correctly uses operator<< to print out the value of s2, but the first check displays the value of s as 0. s should be printed out using boost_test_print_type instead.

ULP Distance and Mollified Tolerance Feature Request

We have a use case for more fine-grained testing of floating point numbers in Boost.Math, and were wondering if there's any interest in adding two new tests to Boost.Test: CHECK_MOLLIFIED_CLOSE and CHECK_ULP_CLOSE. In addition, we are spitting out metadata on test failures (standard representation, hexadecimal representation, ULP distance, and mollified error) which is not currently printed in Boost.Test, and returning a bool on failures that allows more metadata attachment in the testing code.

I have toy implementations of these macros here:

#ifndef BOOST_MATH_TEST_TEST_HPP
#define BOOST_MATH_TEST_TEST_HPP
#include <atomic>
#include <iostream>
#include <iomanip>
#include <boost/assert.hpp>
#include <boost/math/special_functions/next.hpp>
#include <boost/core/demangle.hpp>


namespace boost { namespace math { namespace  test {

namespace detail {
    static std::atomic<size_t> global_error_count = 0;
}

template<class Real>
bool check_mollified_close(Real expected, Real computed, Real tol, std::string const & filename, std::string const & function, int line)
{
    BOOST_ASSERT_MSG(tol >= 0, "Tolerance must be non-negative.");
    using std::max;
    using std::abs;
    Real denom = max(abs(expected), Real(1));
    Real mollified_relative_error = abs(expected - computed)/denom;
    if (mollified_relative_error > tol)
    {
        std::ios_base::fmtflags f( std::cerr.flags() );
        std::cerr << std::setprecision(3);
        std::cerr << "\033[0;31mError at " << filename << ":" << function << ":" << line << ":\n"
                  << " \033[0m Mollified relative error in " << boost::core::demangle(typeid(Real).name())<< " precision is " << mollified_relative_error
                  << ", which exceeds " << tol << ", error/tol  = " << mollified_relative_error/tol << ".\n"
                  << std::setprecision(std::numeric_limits<Real>::digits10) << std::showpos
                  << "  Expected: " << std::defaultfloat << std::fixed << expected << std::hexfloat << " = " << expected << "\n"
                  << "  Computed: " << std::defaultfloat << std::fixed << computed << std::hexfloat << " = " << computed << "\n"
                  << std::defaultfloat
                  << "  ULP distance: " << abs(boost::math::float_distance(expected, computed)) << "\n";
        std::cerr.flags(f);
        ++detail::global_error_count;
        return false;
    }
    return true;
}

template<class Real1, class Real2>
bool check_ulp_close(Real1 expected1, Real2 computed, size_t ulps, std::string const & filename, std::string const & function, int line)
{
    using std::max;
    using std::abs;
    BOOST_ASSERT_MSG(sizeof(Real1) >= sizeof(Real2),
                     "The expected number must be computed in higher (or equal) precision than the number being tested.");

    Real2 expected = Real2(expected1);
    Real2 dist = abs(boost::math::float_distance(expected, computed));
    if (dist > ulps)
    {
        Real2 denom = max(abs(expected), Real2(1));
        Real2 mollified_relative_error = abs(expected - computed)/denom;
        std::ios_base::fmtflags f( std::cerr.flags() );
        std::cerr << std::setprecision(3);
        std::cerr << "\033[0;31mError at " << filename << ":" << function << ":" << line << ":\n"
                  << " \033[0m ULP distance in " << boost::core::demangle(typeid(Real2).name())<< " precision is " << dist
                  << ", which exceeds " << ulps;
                  if (ulps > 0)
                  {
                      std::cerr << ", error/ulps  = " << dist/static_cast<Real2>(ulps) << ".\n";
                  }
                  else
                  {
                      std::cerr << ".\n";
                  }
        std::cerr << std::setprecision(std::numeric_limits<Real2>::digits10) << std::showpos
                  << "  Expected: " << std::defaultfloat << std::fixed << expected << std::hexfloat << " = " << expected << "\n"
                  << "  Computed: " << std::defaultfloat << std::fixed << computed << std::hexfloat << " = " << computed << "\n"
                  << std::defaultfloat
                  << "  Mollified relative error: " << mollified_relative_error << "\n";
        std::cerr.flags(f);
        ++detail::global_error_count;
        return false;
    }
    return true;
}


int report_errors()
{
    if (detail::global_error_count > 0)
    {
        std::cerr << "\033[0;31mError count: " << detail::global_error_count << "\n";
        detail::global_error_count = 0;
        return 1;
    }
    std::cout << "\x1B[32mNo errors detected.\n";
    return 0;
}

}}}

#define CHECK_MOLLIFIED_CLOSE(X, Y, Z) boost::math::test::check_mollified_close< typename std::remove_reference<decltype((Y))>::type>((X), (Y), (Z), __FILE__, __func__, __LINE__)

#define CHECK_ULP_CLOSE(X, Y, Z) boost::math::test::check_ulp_close((X), (Y), (Z), __FILE__, __func__, __LINE__)

#endif

Unreachable code warning on MSVC builds in test matrix

https://www.boost.org/development/tests/develop/output/teeks99-09-p-win2012R2-64on64-test-msvc-14-1-warnings.html#doc-example22

d:\teeks99-09\run\boost_root\boost\test\impl\test_tools.ipp(377) : warning C4702: unreachable code

Happens on most of the tests.

Building something on cygwin (32 and 64-bit) that uses Boost.Test requires cxxstd-dialect=gnu

I've been adding CI environments to the CMT projects and there's a common problem I have found with cygwin targets on appveyor, they show up like this:

gcc.compile.c++ bin.v2\libs\property_map\test\compose_property_map_test.test\gcc-6.4.0\release\cxxstd-03-iso\target-os-cygwin\compose_property_map_test.o
In file included from ./boost/test/minimal.hpp:54:0,
                 from libs\property_map\test\compose_property_map_test.cpp:11:
./boost/test/impl/debug.ipp: In function 'const char* boost::debug::{anonymous}::prepare_gdb_cmnd_file(const boost::debug::dbg_startup_info&)':
./boost/test/impl/debug.ipp:424:23: error: '::mkstemp' has not been declared
     fd_holder cmd_fd( ::mkstemp( cmd_file_name ) );
                       ^~
In file included from ./boost/test/minimal.hpp:54:0,
                 from libs\property_map\test\compose_property_map_test.cpp:11:
./boost/test/impl/debug.ipp: In function 'void boost::debug::debugger_break()':
./boost/test/impl/debug.ipp:711:5: error: '::kill' has not been declared
     ::kill( ::getpid(), SIGTRAP );
     ^~
./boost/test/impl/debug.ipp: In function 'bool boost::debug::attach_debugger(bool)':
./boost/test/impl/debug.ipp:892:34: error: '::mkstemp' has not been declared
     fd_holder init_done_lock_fd( ::mkstemp( init_done_lock_fn ) );
                                  ^~
./boost/test/impl/debug.ipp:929:9: error: '::select' has not been declared
         ::select( 0, 0, 0, 0, &to );
         ^~

It looks like missing includes, perhaps.
Here's a build job showing it.
https://ci.appveyor.com/project/jeking3/property-map/build/1.0.2-develop/job/9tt6av7lxe14qy8h

Is this a known issue?

massive performance regression between 1.63 and 1.67

Hi,

I haven't been able to bisect yet but I have observed a massive performance degradation between 1.63 and 1.67 when using a BOOST_DATA_TEST_CASE with an xrange of 100,000 values.

It takes 10 minutes just for the test system to initialize itself. It was instant before.

"master_test_suite" declared twice

In file included from /usr/local/include/boost/test/tree/auto_registration.hpp:21,
                 from /usr/local/include/boost/test/unit_test_suite.hpp:17,
                 from /usr/local/include/boost/test/unit_test.hpp:19,
                 from test/test_bitcoin_main.cpp:11:
/usr/local/include/boost/test/tree/test_unit.hpp:46:38: warning: redundant redeclaration of 'boost::unit_test::master_test_suite_t& boost::unit_test::framework::master_test_suite()' in same scope [-Wredundant-decls]
 BOOST_TEST_DECL master_test_suite_t& master_test_suite();
                                      ^~~~~~~~~~~~~~~~~
In file included from /usr/local/include/boost/test/unit_test_suite.hpp:16,
                 from /usr/local/include/boost/test/unit_test.hpp:19,
                 from test/test_bitcoin_main.cpp:11:
/usr/local/include/boost/test/framework.hpp:195:38: note: previous declaration of 'boost::unit_test::master_test_suite_t& boost::unit_test::framework::master_test_suite()'
 BOOST_TEST_DECL master_test_suite_t& master_test_suite();

Should I submit a patch?

Support BOOST_UNIT_TEST_FRAMEWORK_DYN_LINK et al

The CMake config files that b2 install installs automatically define BOOST_${LIBNAME}_DYN_LINK=1 when the user links to the dynamic library $LIBNAME, by means of linking to the Boost::$libname target. This works most of the time, because the target/library name and the macro match, but doesn't for Boost::unit_test_framework, Boost::test_exec_monitor and Boost::prg_exec_monitor, because these use the macro BOOST_TEST_DYN_LINK and this doesn't match the target names.

So, would it be possible to make the libraries recognize their individual macros in addition to BOOST_TEST_DYN_LINK?

Adding several test-suites with same name

See https://svn.boost.org/trac10/ticket/12597

expected_failures doesnt work for BOOST_DATA_TEST_CASE

BOOST_TEST_DECORATOR(*boost::unit_test::decorator::expected_failures(5))
BOOST_DATA_TEST_CASE(test1, boost::unit_test::data::make({1}), digit) {
    BOOST_CHECK(0);
}

Output:

Running 1 test case...

0%   10   20   30   40   50   60   70   80   90   100%
|----|----|----|----|----|----|----|----|----|----|
path/to/file.cpp(206): error: in "test1/_0": check 0 has failed
Failure occurred in a following context:
    digit = 1;
***************************************************

*** 1 failure is detected (11 failures are expected) in the test module "Common"

Feature Request: Add ability to skip test cases being inside them

Hi!

I want to use something similar to skip() method in python unittest: https://docs.python.org/2/library/unittest.html#skipping-tests-and-expected-failures .
The goal is to check at runtime whether this test case should be run (or should be ignored in the results) or not. For example, if some condition is true we can do BOOST_SKIP("There is no enough free disk space on your computer") and this test case will not be counted as "failed". Instead, probably there can be another column "skipped" which can collect statistics about skipped test cases.

Plan B (the simplest workaround): Is to allow to add expected_failure decorator to a test, being inside the test. For example, in this case we can do something like:

if (...) {
  current_test_case()->add_decorator(*expected_failures(1));
  BOOST_FAIL("There is no enough free disk space on your computer");
}

[snippet_dataset1_3] seems to be broken

There's a link to [snippet_dataset1_3]:

test/doc/test_organization/parametric_test_case_generation.qbk

Lines 193 to 196 in 2b1e4cd

 The third form is an extension of the previous form for datasets of arity `N`. The sample being a polymorphic tuple, each 

 of the variables `var1`, ..., `varN` corresponds to the index 1, ... `N` of the the sample: 

 [snippet_dataset1_3]

Here is the snippet:

test/doc/snippet/dataset_1/test_file.cpp

Lines 38 to 44 in 2b1e4cd

 //[snippet_dataset1_3 

 BOOST_DATA_TEST_CASE( test_case_arity2, data::xrange(2) ^ data::xrange(5), apples, potatoes) 

 { 

 BOOST_TEST((apples <= 1 && apples >= 0)); 

 BOOST_TEST((potatoes <= 4 && potatoes >= 0)); 

 } 

 //]

But according to this

test/doc/test_organization/parametric_test_case_generation.qbk

Lines 306 to 310 in 2b1e4cd

 A particular handling is performed if `dsa` and `dsb` are of different size. The rule is as follow: 

 * if the both zipped datasets have the same size, this is the size of the resulting dataset (this size can then be infinite). 

 * otherwise if one of the dataset is of size 1 (singleton) or of infinite size, the resulting size is governed by the other dataset. 

 * otherwise an exception is thrown at runtime

the snippet must throw. And it does throw (msvc 19.15.26732.1 for x86 + boost.test 1.67.0).

So is it a broken snippet? I didn't find any mentions in the docs this to be intentional behavior.

doc_example22 (and 23) are expected to fail, but do not on clang with release variant builds

Clang 6, cxxstd=2a: https://travis-ci.org/jeking3/test/jobs/453349509#L3563
Clang 5, cxxstd=17: https://travis-ci.org/jeking3/test/jobs/453349504#L3450
Clang 4, cxxstd=14: https://travis-ci.org/jeking3/test/jobs/453349500#L3418

boost@82aef92a4871:/boost/libs/test/test$ clang++-6.0 --version
clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

boost@82aef92a4871:/boost/libs/test/test$ ../../../b2 variant=release cxxstd=17 toolset=clang doc-example22
Performing configuration checks

    - BOOST_COMP_GNUC >= 4.3.0 : no
    - default address-model    : 64-bit (cached)
    - default architecture     : x86 (cached)
    - symlinks supported       : yes (cached)
    - Boost.Config Feature Check: cxx11_decltype : yes (cached)
    - Boost.Config Feature Check: cxx11_hdr_random : yes (cached)
    - Boost.Config Feature Check: cxx11_hdr_tuple : yes (cached)
    - Boost.Config Feature Check: cxx11_hdr_initializer_list : yes (cached)
    - Boost.Config Feature Check: cxx11_variadic_macros : yes (cached)
    - Boost.Config Feature Check: cxx11_template_aliases : yes (cached)
    - BOOST_COMP_GNUC >= 4.3.0 : no  (cached)
    - Boost.Config Feature Check: cxx11_decltype : yes (cached)
    - Boost.Config Feature Check: cxx11_hdr_random : yes (cached)
    - Boost.Config Feature Check: cxx11_hdr_tuple : yes (cached)
    - Boost.Config Feature Check: cxx11_hdr_initializer_list : yes (cached)
    - Boost.Config Feature Check: cxx11_variadic_macros : yes (cached)
    - Boost.Config Feature Check: cxx11_template_aliases : yes (cached)
...found 1990 targets...
...updating 13 targets...
clang-linux.compile.c++.without-pth ../../../bin.v2/libs/test/test/doc-example22.test/clang-linux-6.0/release/cxxstd-17-iso/visibility-hidden/example22.run-fail.o
clang-linux.compile.c++.without-pth ../../../bin.v2/libs/system/build/clang-linux-6.0/release/cxxstd-17-iso/visibility-hidden/error_code.o
clang-linux.link.dll ../../../bin.v2/libs/system/build/clang-linux-6.0/release/cxxstd-17-iso/visibility-hidden/libboost_system.so.1.69.0
clang-linux.link ../../../bin.v2/libs/test/test/doc-example22.test/clang-linux-6.0/release/cxxstd-17-iso/visibility-hidden/doc-example22
testing.capture-output ../../../bin.v2/libs/test/test/doc-example22.test/clang-linux-6.0/release/cxxstd-17-iso/visibility-hidden/doc-example22.run
...failed testing.capture-output ../../../bin.v2/libs/test/test/doc-example22.test/clang-linux-6.0/release/cxxstd-17-iso/visibility-hidden/doc-example22.run...
...removing ../../../bin.v2/libs/test/test/doc-example22.test/clang-linux-6.0/release/cxxstd-17-iso/visibility-hidden/doc-example22.run
...failed updating 1 target...
...skipped 2 targets...
...updated 10 targets...

boost@82aef92a4871:/boost/libs/test/test$ ../../../bin.v2/libs/test/test/doc-example22.test/clang-linux-6.0/release/cxxstd-17-iso/visibility-hidden/doc-example
22
Running 1 test case...

*** No errors detected

Detecting boost_test_print_type does not work when testing a type with an explicit conversion to bool

struct s {
	explicit operator bool() const { return false; }
};
std::ostream &boost_test_print_type(std::ostream &o, const s &) { return o << 's'; }

When testing a value of type s (e.g. BOOST_TEST(s{})), compilation fails because the compiler is looking for an overload of operator<<, but we haven't provided one. As per the documentation, providing a definition of boost_test_print_type should be sufficient.

Catch block for boost::exception appears after std::exception in execution_monitor::execute()

So, if I throw myexception defined as struct myexception : std::exception, virtual boost::exception it prints uninformative message

TSan reports data races in unit_test_log_impl::set_checkpoint

On lines

test/include/boost/test/impl/unit_test_log.ipp

Lines 151 to 152 in 90d577a

 m_checkpoint_data.m_file_name = file; 

 m_checkpoint_data.m_line_num = line_num;

The report

==================
WARNING: ThreadSanitizer: data race (pid=63609)
  Write of size 8 at 0x7fa187ab60f8 by thread T2:
    #0 boost::unit_test::basic_cstring<char const>::operator=(boost::unit_test::basic_cstring<char const> const&) /home/raven/boost-repo/./boost/test/utils/basic_cstring/basic_cstring.hpp:402:13 (libboost_unit_test_framework.so.1.69.0+0x338fd)
    #1 boost::unit_test::(anonymous namespace)::unit_test_log_impl::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:151:41 (libboost_unit_test_framework.so.1.69.0+0xcde85)
    #2 boost::unit_test::unit_test_log_t::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:302:18 (libboost_unit_test_framework.so.1.69.0+0xcddc9)
    #3 do_test_tss() /home/raven/boost-repo/libs/thread/test/test_tss.cpp:180:5 (test_tss+0x4cabf4)
    #4 boost::detail::thread_data<void (*)()>::run() /home/raven/boost-repo/./boost/thread/detail/thread.hpp:117:17 (test_tss+0x4e68c6)
    #5 boost::(anonymous namespace)::thread_proxy(void*) /home/raven/boost-repo/libs/thread/src/pthread/thread.cpp:177:34 (libboost_thread.so.1.69.0+0x10e52)

  Previous write of size 8 at 0x7fa187ab60f8 by main thread:
    #0 boost::unit_test::basic_cstring<char const>::operator=(boost::unit_test::basic_cstring<char const> const&) /home/raven/boost-repo/./boost/test/utils/basic_cstring/basic_cstring.hpp:402:13 (libboost_unit_test_framework.so.1.69.0+0x338fd)
    #1 boost::unit_test::(anonymous namespace)::unit_test_log_impl::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:151:41 (libboost_unit_test_framework.so.1.69.0+0xcde85)
    #2 boost::unit_test::unit_test_log_t::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:302:18 (libboost_unit_test_framework.so.1.69.0+0xcddc9)
    #3 void (anonymous namespace)::timed_test<void (*)()>(void (*)(), int, (anonymous namespace)::execution_monitor::wait_type) /home/raven/boost-repo/libs/thread/test/./util.inl:159:5 (test_tss+0x4cbe65)
    #4 test_tss::test_method() /home/raven/boost-repo/libs/thread/test/test_tss.cpp:214:5 (test_tss+0x4cbd24)
    #5 test_tss_invoker() /home/raven/boost-repo/libs/thread/test/test_tss.cpp:212:1 (test_tss+0x4cb90a)
    #6 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /home/raven/boost-repo/./boost/function/function_template.hpp:118:11 (test_tss+0x4ea5cd)
    #7 boost::function0<void>::operator()() const /home/raven/boost-repo/./boost/function/function_template.hpp:768:14 (libboost_unit_test_framework.so.1.69.0+0x576b8)
    #8 boost::detail::forward::operator()() /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:1312:32 (libboost_unit_test_framework.so.1.69.0+0x575d8)
    #9 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) /home/raven/boost-repo/./boost/function/function_template.hpp:138:18 (libboost_unit_test_framework.so.1.69.0+0x571a0)
    #10 boost::function0<int>::operator()() const /home/raven/boost-repo/./boost/function/function_template.hpp:768:14 (libboost_unit_test_framework.so.1.69.0+0x50458)
    #11 int boost::detail::do_invoke<boost::shared_ptr<boost::detail::translator_holder_base>, boost::function<int ()> >(boost::shared_ptr<boost::detail::translator_holder_base> const&, boost::function<int ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:286:30 (libboost_unit_test_framework.so.1.69.0+0x4e899)
    #12 boost::execution_monitor::catch_signals(boost::function<int ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:875:16 (libboost_unit_test_framework.so.1.69.0+0x4bdd3)
    #13 boost::execution_monitor::execute(boost::function<int ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:1214:16 (libboost_unit_test_framework.so.1.69.0+0x4c061)
    #14 boost::execution_monitor::vexecute(boost::function<void ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:1321:5 (libboost_unit_test_framework.so.1.69.0+0x4de67)
    #15 boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned int) /home/raven/boost-repo/./boost/test/impl/unit_test_monitor.ipp:49:9 (libboost_unit_test_framework.so.1.69.0+0xdc1b7)
    #16 boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) /home/raven/boost-repo/./boost/test/impl/framework.ipp:789:44 (libboost_unit_test_framework.so.1.69.0+0x694f5)
    #17 boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) /home/raven/boost-repo/./boost/test/impl/framework.ipp:737:54 (libboost_unit_test_framework.so.1.69.0+0x68c47)
    #18 boost::unit_test::framework::run(unsigned long, bool) /home/raven/boost-repo/./boost/test/impl/framework.ipp:1631:29 (libboost_unit_test_framework.so.1.69.0+0x5d5c5)
    #19 boost::unit_test::unit_test_main(bool (*)(), int, char**) /home/raven/boost-repo/./boost/test/impl/unit_test_main.ipp:247:9 (libboost_unit_test_framework.so.1.69.0+0xd8977)
    #20 main /home/raven/boost-repo/./boost/test/unit_test.hpp:63:12 (test_tss+0x4ca0f2)

  Location is global 'boost::unit_test::(anonymous namespace)::s_log_impl()::the_inst' of size 128 at 0x7fa187ab60b0 (libboost_unit_test_framework.so.1.69.0+0x0000003bf0f8)

  Thread T2 (tid=63612, running) created by thread T1 at:
    #0 pthread_create <null> (test_tss+0x432846)
    #1 boost::thread::start_thread_noexcept() /home/raven/boost-repo/libs/thread/src/pthread/thread.cpp:261:25 (libboost_thread.so.1.69.0+0x10d61)
    #2 boost::thread::start_thread() /home/raven/boost-repo/./boost/thread/detail/thread.hpp:179:16 (test_tss+0x4e62ab)
    #3 boost::thread::thread<void (* const&)()>(void (* const&)()) /home/raven/boost-repo/./boost/thread/detail/thread.hpp:266:13 (test_tss+0x4eb6f8)
    #4 thread_detail_anon::indirect_adapter<void (*)()>::operator()() const /home/raven/boost-repo/libs/thread/test/./util.inl:129:27 (test_tss+0x4eb587)
    #5 boost::detail::thread_data<thread_detail_anon::indirect_adapter<void (*)()> >::run() /home/raven/boost-repo/./boost/thread/detail/thread.hpp:117:17 (test_tss+0x4eb51f)
    #6 boost::(anonymous namespace)::thread_proxy(void*) /home/raven/boost-repo/libs/thread/src/pthread/thread.cpp:177:34 (libboost_thread.so.1.69.0+0x10e52)

SUMMARY: ThreadSanitizer: data race /home/raven/boost-repo/./boost/test/utils/basic_cstring/basic_cstring.hpp:402:13 in boost::unit_test::basic_cstring<char const>::operator=(boost::unit_test::basic_cstring<char const> const&)
==================
==================
WARNING: ThreadSanitizer: data race (pid=63609)
  Write of size 8 at 0x7fa187ab6100 by thread T2:
    #0 boost::unit_test::basic_cstring<char const>::operator=(boost::unit_test::basic_cstring<char const> const&) /home/raven/boost-repo/./boost/test/utils/basic_cstring/basic_cstring.hpp:403:13 (libboost_unit_test_framework.so.1.69.0+0x3393b)
    #1 boost::unit_test::(anonymous namespace)::unit_test_log_impl::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:151:41 (libboost_unit_test_framework.so.1.69.0+0xcde85)
    #2 boost::unit_test::unit_test_log_t::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:302:18 (libboost_unit_test_framework.so.1.69.0+0xcddc9)
    #3 do_test_tss() /home/raven/boost-repo/libs/thread/test/test_tss.cpp:180:5 (test_tss+0x4cabf4)
    #4 boost::detail::thread_data<void (*)()>::run() /home/raven/boost-repo/./boost/thread/detail/thread.hpp:117:17 (test_tss+0x4e68c6)
    #5 boost::(anonymous namespace)::thread_proxy(void*) /home/raven/boost-repo/libs/thread/src/pthread/thread.cpp:177:34 (libboost_thread.so.1.69.0+0x10e52)

  Previous write of size 8 at 0x7fa187ab6100 by main thread:
    #0 boost::unit_test::basic_cstring<char const>::operator=(boost::unit_test::basic_cstring<char const> const&) /home/raven/boost-repo/./boost/test/utils/basic_cstring/basic_cstring.hpp:403:13 (libboost_unit_test_framework.so.1.69.0+0x3393b)
    #1 boost::unit_test::(anonymous namespace)::unit_test_log_impl::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:151:41 (libboost_unit_test_framework.so.1.69.0+0xcde85)
    #2 boost::unit_test::unit_test_log_t::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:302:18 (libboost_unit_test_framework.so.1.69.0+0xcddc9)
    #3 void (anonymous namespace)::timed_test<void (*)()>(void (*)(), int, (anonymous namespace)::execution_monitor::wait_type) /home/raven/boost-repo/libs/thread/test/./util.inl:159:5 (test_tss+0x4cbe65)
    #4 test_tss::test_method() /home/raven/boost-repo/libs/thread/test/test_tss.cpp:214:5 (test_tss+0x4cbd24)
    #5 test_tss_invoker() /home/raven/boost-repo/libs/thread/test/test_tss.cpp:212:1 (test_tss+0x4cb90a)
    #6 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /home/raven/boost-repo/./boost/function/function_template.hpp:118:11 (test_tss+0x4ea5cd)
    #7 boost::function0<void>::operator()() const /home/raven/boost-repo/./boost/function/function_template.hpp:768:14 (libboost_unit_test_framework.so.1.69.0+0x576b8)
    #8 boost::detail::forward::operator()() /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:1312:32 (libboost_unit_test_framework.so.1.69.0+0x575d8)
    #9 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) /home/raven/boost-repo/./boost/function/function_template.hpp:138:18 (libboost_unit_test_framework.so.1.69.0+0x571a0)
    #10 boost::function0<int>::operator()() const /home/raven/boost-repo/./boost/function/function_template.hpp:768:14 (libboost_unit_test_framework.so.1.69.0+0x50458)
    #11 int boost::detail::do_invoke<boost::shared_ptr<boost::detail::translator_holder_base>, boost::function<int ()> >(boost::shared_ptr<boost::detail::translator_holder_base> const&, boost::function<int ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:286:30 (libboost_unit_test_framework.so.1.69.0+0x4e899)
    #12 boost::execution_monitor::catch_signals(boost::function<int ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:875:16 (libboost_unit_test_framework.so.1.69.0+0x4bdd3)
    #13 boost::execution_monitor::execute(boost::function<int ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:1214:16 (libboost_unit_test_framework.so.1.69.0+0x4c061)
    #14 boost::execution_monitor::vexecute(boost::function<void ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:1321:5 (libboost_unit_test_framework.so.1.69.0+0x4de67)
    #15 boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned int) /home/raven/boost-repo/./boost/test/impl/unit_test_monitor.ipp:49:9 (libboost_unit_test_framework.so.1.69.0+0xdc1b7)
    #16 boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) /home/raven/boost-repo/./boost/test/impl/framework.ipp:789:44 (libboost_unit_test_framework.so.1.69.0+0x694f5)
    #17 boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) /home/raven/boost-repo/./boost/test/impl/framework.ipp:737:54 (libboost_unit_test_framework.so.1.69.0+0x68c47)
    #18 boost::unit_test::framework::run(unsigned long, bool) /home/raven/boost-repo/./boost/test/impl/framework.ipp:1631:29 (libboost_unit_test_framework.so.1.69.0+0x5d5c5)
    #19 boost::unit_test::unit_test_main(bool (*)(), int, char**) /home/raven/boost-repo/./boost/test/impl/unit_test_main.ipp:247:9 (libboost_unit_test_framework.so.1.69.0+0xd8977)
    #20 main /home/raven/boost-repo/./boost/test/unit_test.hpp:63:12 (test_tss+0x4ca0f2)

  Location is global 'boost::unit_test::(anonymous namespace)::s_log_impl()::the_inst' of size 128 at 0x7fa187ab60b0 (libboost_unit_test_framework.so.1.69.0+0x0000003bf100)

  Thread T2 (tid=63612, running) created by thread T1 at:
    #0 pthread_create <null> (test_tss+0x432846)
    #1 boost::thread::start_thread_noexcept() /home/raven/boost-repo/libs/thread/src/pthread/thread.cpp:261:25 (libboost_thread.so.1.69.0+0x10d61)
    #2 boost::thread::start_thread() /home/raven/boost-repo/./boost/thread/detail/thread.hpp:179:16 (test_tss+0x4e62ab)
    #3 boost::thread::thread<void (* const&)()>(void (* const&)()) /home/raven/boost-repo/./boost/thread/detail/thread.hpp:266:13 (test_tss+0x4eb6f8)
    #4 thread_detail_anon::indirect_adapter<void (*)()>::operator()() const /home/raven/boost-repo/libs/thread/test/./util.inl:129:27 (test_tss+0x4eb587)
    #5 boost::detail::thread_data<thread_detail_anon::indirect_adapter<void (*)()> >::run() /home/raven/boost-repo/./boost/thread/detail/thread.hpp:117:17 (test_tss+0x4eb51f)
    #6 boost::(anonymous namespace)::thread_proxy(void*) /home/raven/boost-repo/libs/thread/src/pthread/thread.cpp:177:34 (libboost_thread.so.1.69.0+0x10e52)

SUMMARY: ThreadSanitizer: data race /home/raven/boost-repo/./boost/test/utils/basic_cstring/basic_cstring.hpp:403:13 in boost::unit_test::basic_cstring<char const>::operator=(boost::unit_test::basic_cstring<char const> const&)
==================
==================
WARNING: ThreadSanitizer: data race (pid=63609)
  Write of size 8 at 0x7fa187ab6108 by thread T2:
    #0 boost::unit_test::(anonymous namespace)::unit_test_log_impl::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:152:41 (libboost_unit_test_framework.so.1.69.0+0xcdea2)
    #1 boost::unit_test::unit_test_log_t::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:302:18 (libboost_unit_test_framework.so.1.69.0+0xcddc9)
    #2 do_test_tss() /home/raven/boost-repo/libs/thread/test/test_tss.cpp:180:5 (test_tss+0x4cabf4)
    #3 boost::detail::thread_data<void (*)()>::run() /home/raven/boost-repo/./boost/thread/detail/thread.hpp:117:17 (test_tss+0x4e68c6)
    #4 boost::(anonymous namespace)::thread_proxy(void*) /home/raven/boost-repo/libs/thread/src/pthread/thread.cpp:177:34 (libboost_thread.so.1.69.0+0x10e52)

  Previous write of size 8 at 0x7fa187ab6108 by main thread:
    #0 boost::unit_test::(anonymous namespace)::unit_test_log_impl::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:152:41 (libboost_unit_test_framework.so.1.69.0+0xcdea2)
    #1 boost::unit_test::unit_test_log_t::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>) /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:302:18 (libboost_unit_test_framework.so.1.69.0+0xcddc9)
    #2 void (anonymous namespace)::timed_test<void (*)()>(void (*)(), int, (anonymous namespace)::execution_monitor::wait_type) /home/raven/boost-repo/libs/thread/test/./util.inl:159:5 (test_tss+0x4cbe65)
    #3 test_tss::test_method() /home/raven/boost-repo/libs/thread/test/test_tss.cpp:214:5 (test_tss+0x4cbd24)
    #4 test_tss_invoker() /home/raven/boost-repo/libs/thread/test/test_tss.cpp:212:1 (test_tss+0x4cb90a)
    #5 boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /home/raven/boost-repo/./boost/function/function_template.hpp:118:11 (test_tss+0x4ea5cd)
    #6 boost::function0<void>::operator()() const /home/raven/boost-repo/./boost/function/function_template.hpp:768:14 (libboost_unit_test_framework.so.1.69.0+0x576b8)
    #7 boost::detail::forward::operator()() /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:1312:32 (libboost_unit_test_framework.so.1.69.0+0x575d8)
    #8 boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) /home/raven/boost-repo/./boost/function/function_template.hpp:138:18 (libboost_unit_test_framework.so.1.69.0+0x571a0)
    #9 boost::function0<int>::operator()() const /home/raven/boost-repo/./boost/function/function_template.hpp:768:14 (libboost_unit_test_framework.so.1.69.0+0x50458)
    #10 int boost::detail::do_invoke<boost::shared_ptr<boost::detail::translator_holder_base>, boost::function<int ()> >(boost::shared_ptr<boost::detail::translator_holder_base> const&, boost::function<int ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:286:30 (libboost_unit_test_framework.so.1.69.0+0x4e899)
    #11 boost::execution_monitor::catch_signals(boost::function<int ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:875:16 (libboost_unit_test_framework.so.1.69.0+0x4bdd3)
    #12 boost::execution_monitor::execute(boost::function<int ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:1214:16 (libboost_unit_test_framework.so.1.69.0+0x4c061)
    #13 boost::execution_monitor::vexecute(boost::function<void ()> const&) /home/raven/boost-repo/./boost/test/impl/execution_monitor.ipp:1321:5 (libboost_unit_test_framework.so.1.69.0+0x4de67)
    #14 boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned int) /home/raven/boost-repo/./boost/test/impl/unit_test_monitor.ipp:49:9 (libboost_unit_test_framework.so.1.69.0+0xdc1b7)
    #15 boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) /home/raven/boost-repo/./boost/test/impl/framework.ipp:789:44 (libboost_unit_test_framework.so.1.69.0+0x694f5)
    #16 boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned int, boost::unit_test::framework::state::random_generator_helper const*) /home/raven/boost-repo/./boost/test/impl/framework.ipp:737:54 (libboost_unit_test_framework.so.1.69.0+0x68c47)
    #17 boost::unit_test::framework::run(unsigned long, bool) /home/raven/boost-repo/./boost/test/impl/framework.ipp:1631:29 (libboost_unit_test_framework.so.1.69.0+0x5d5c5)
    #18 boost::unit_test::unit_test_main(bool (*)(), int, char**) /home/raven/boost-repo/./boost/test/impl/unit_test_main.ipp:247:9 (libboost_unit_test_framework.so.1.69.0+0xd8977)
    #19 main /home/raven/boost-repo/./boost/test/unit_test.hpp:63:12 (test_tss+0x4ca0f2)

  Location is global 'boost::unit_test::(anonymous namespace)::s_log_impl()::the_inst' of size 128 at 0x7fa187ab60b0 (libboost_unit_test_framework.so.1.69.0+0x0000003bf108)

  Thread T2 (tid=63612, running) created by thread T1 at:
    #0 pthread_create <null> (test_tss+0x432846)
    #1 boost::thread::start_thread_noexcept() /home/raven/boost-repo/libs/thread/src/pthread/thread.cpp:261:25 (libboost_thread.so.1.69.0+0x10d61)
    #2 boost::thread::start_thread() /home/raven/boost-repo/./boost/thread/detail/thread.hpp:179:16 (test_tss+0x4e62ab)
    #3 boost::thread::thread<void (* const&)()>(void (* const&)()) /home/raven/boost-repo/./boost/thread/detail/thread.hpp:266:13 (test_tss+0x4eb6f8)
    #4 thread_detail_anon::indirect_adapter<void (*)()>::operator()() const /home/raven/boost-repo/libs/thread/test/./util.inl:129:27 (test_tss+0x4eb587)
    #5 boost::detail::thread_data<thread_detail_anon::indirect_adapter<void (*)()> >::run() /home/raven/boost-repo/./boost/thread/detail/thread.hpp:117:17 (test_tss+0x4eb51f)
    #6 boost::(anonymous namespace)::thread_proxy(void*) /home/raven/boost-repo/libs/thread/src/pthread/thread.cpp:177:34 (libboost_thread.so.1.69.0+0x10e52)

SUMMARY: ThreadSanitizer: data race /home/raven/boost-repo/./boost/test/impl/unit_test_log.ipp:152:41 in boost::unit_test::(anonymous namespace)::unit_test_log_impl::set_checkpoint(boost::unit_test::basic_cstring<char const>, unsigned long, boost::unit_test::basic_cstring<char const>)
==================

Runtime type_mismatch after upgrade to 1.69

Hello,

after my distribution (Arch) rolled out the upgrade to Boost 1.69 I am unable to compile my application anymore.

I try to get the log_level set. Minimal compiling example:

#define BOOST_TEST_DYN_LINK
#include <boost/test/unit_test.hpp>
#include <boost/test/unit_test_parameters.hpp>

bool init_unit_test()
{
  using namespace boost::unit_test;
  
  auto logLevel = runtime_config::get<log_level>(runtime_config::btrt_log_level);
  return true;
}

int main(int argc, char* argv[])
{
  int retCode = boost::unit_test::unit_test_main( &init_unit_test, argc, argv );
  return retCode;
}

which compiles fine with g++ -std=c++11 -lboost_unit_test_framework boosttesting.cpp or with clang++

but gives an runtime error (exact same for both compilers):

 % ./a.out 
Test setup error: boost::runtime::arg_type_mismatch: Access with invalid type for argument corresponding to parameter log_level

% g++ --version
g++ (GCC) 8.2.1 20181127

% clang++ --version
clang version 7.0.1 (tags/RELEASE_701/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Thanks!

Compile time error with Cygwins and boost/test/minimal.hpp

Attempt to build Boost.Variant tests under Cygwin gives the following error:

In file included from ..\..\../boost/test/minimal.hpp:54:0,
                 from rvalue_test.cpp:14:
..\..\../boost/test/impl/debug.ipp: In function 'const char* boost::debug::{anonymous}::prepare_gdb_cmnd_file(const boost::debug::dbg_startup_info&)':
..\..\../boost/test/impl/debug.ipp:424:23: error: '::mkstemp' has not been declared
     fd_holder cmd_fd( ::mkstemp( cmd_file_name ) );
                       ^~
In file included from ..\..\../boost/test/minimal.hpp:54:0,
                 from rvalue_test.cpp:14:
..\..\../boost/test/impl/debug.ipp: In function 'void boost::debug::debugger_break()':
..\..\../boost/test/impl/debug.ipp:711:5: error: '::kill' has not been declared
     ::kill( ::getpid(), SIGTRAP );
     ^~
..\..\../boost/test/impl/debug.ipp: In function 'bool boost::debug::attach_debugger(bool)':
..\..\../boost/test/impl/debug.ipp:892:34: error: '::mkstemp' has not been declared
     fd_holder init_done_lock_fd( ::mkstemp( init_done_lock_fn ) );
                                  ^~
..\..\../boost/test/impl/debug.ipp:929:9: error: '::select' has not been declared
         ::select( 0, 0, 0, 0, &to );
         ^~

Whole build logs are available here: https://ci.appveyor.com/project/apolukhin/variant/build/job/q0all7ghv0t975rv

Undefined symbols for architecture x86_64: "vtable for boost::unit_test::unit_test_log_t"

I can use many Boost libs, but always failed with boost/test while building with Xcode on MacOS
MACOSX_DEPLOYMENT_TARGET=10.13
issue message:
Undefined symbols for architecture x86_64:
"vtable for boost::unit_test::unit_test_log_t", referenced from:
boost::unit_test::unit_test_log_t::unit_test_log_t() in main.o
NOTE: a missing vtable usually means the first non-inline virtual member function has no definition.
ld: symbol(s) not found for architecture x86_64

UBSAN while building a test in Boost.Thread (stack is in Test)

Output: https://travis-ci.org/jeking3/thread/jobs/457490782#L2801

gcc.link ../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept
testing.capture-output ../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.run
====== BEGIN OUTPUT ======
../../boost/test/impl/test_tree.ipp:273:25: runtime error: member call on address 0x7ffcfb300370 which does not point to an object of type 'test_unit_generator'
0x7ffcfb300370: note: object is of type 'boost::unit_test::ut_detail::template_test_case_gen<test_scoped_lock_concept_invoker, boost::mpl::vector<boost::mutex, boost::timed_mutex, boost::recursive_mutex, boost::recursive_timed_mutex, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >'
 00 00 00 00  98 8c 50 00 00 00 00 00  e0 1b d9 01 00 00 00 00  b0 23 d9 01 00 00 00 00  20 65 b6 9e
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'boost::unit_test::ut_detail::template_test_case_gen<test_scoped_lock_concept_invoker, boost::mpl::vector<boost::mutex, boost::timed_mutex, boost::recursive_mutex, boost::recursive_timed_mutex, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >'
    #0 0x7fbe9e6349b1 in boost::unit_test::test_suite::add(boost::unit_test::test_unit_generator const&, boost::unit_test::decorator::collector_t&) ../../boost/test/impl/test_tree.ipp:273
    #1 0x7fbe9e637f0f in boost::unit_test::ut_detail::auto_test_unit_registrar::auto_test_unit_registrar(boost::unit_test::test_unit_generator const&, boost::unit_test::decorator::collector_t&) ../../boost/test/impl/test_tree.ipp:509
    #2 0x40d369 in __static_initialization_and_destruction_0 ../../libs/thread/test/test_lock_concept.cpp:446
    #3 0x40dab4 in _GLOBAL__sub_I_test_lock_concept.cpp ../../libs/thread/test/test_lock_concept.cpp:575
    #4 0x4e506c in __libc_csu_init (/home/travis/build/jeking3/boost-root/bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept+0x4e506c)
    #5 0x7fbe9c8e7ed4 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21ed4)
    #6 0x40b288  (/home/travis/build/jeking3/boost-root/bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept+0x40b288)
EXIT STATUS: 1
====== END OUTPUT ======
    LD_LIBRARY_PATH="/home/travis/build/jeking3/boost-root/bin.v2/libs/chrono/build/gcc-8/debug/cxxstd-03-iso/threading-multi/visibility-hidden:/home/travis/build/jeking3/boost-root/bin.v2/libs/system/build/gcc-8/debug/cxxstd-03-iso/threading-multi/visibility-hidden:/home/travis/build/jeking3/boost-root/bin.v2/libs/test/build/gcc-8/debug/cxxstd-03-iso/threading-multi/visibility-hidden:/home/travis/build/jeking3/boost-root/bin.v2/libs/thread/build/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden:/home/travis/build/jeking3/boost-root/bin.v2/libs/timer/build/gcc-8/debug/cxxstd-03-iso/threading-multi/visibility-hidden:/usr/bin:/usr/lib:/usr/lib32:/usr/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
    status=0
    if test $status -ne 0 ; then
        echo Skipping test execution due to testing.execute=off
        exit 0
    fi
     "../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept"   > "../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output" 2>&1 < /dev/null
    status=$?
    echo >> "../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output"
    echo EXIT STATUS: $status >> "../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output"
    if test $status -eq 0 ; then
        cp "../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output" "../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.run"
    fi
    verbose=0
    if test $status -ne 0 ; then
        verbose=1
    fi
    if test $verbose -eq 1 ; then
        echo ====== BEGIN OUTPUT ======
        cat "../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output"
        echo ====== END OUTPUT ======
    fi
    exit $status
...failed testing.capture-output ../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/cxxstd-03-iso/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.run...

Discovered in Travis CI; Reproducible in the Boost Docker Development Environment:

jking@ubuntu:~/boost/libs/thread/test$ bdde
docker run -v /home/jking/boost:/boost:rw -v /home/jking/bdde:/bdde:ro --workdir /boost/libs/thread/test -it jeking3/bdde:linux /bin/bash
boost@cbc691077d92:/boost/libs/thread/test$ ubsan test_lock_concept
UBSAN_OPTIONS=print_stacktrace=1 b2 define=BOOST_NO_STRESS_TEST=1 cxxflags=-fno-omit-frame-pointer cxxflags=-fsanitize=undefined cxxflags=-fno-sanitize-recover=undefined linkflags=-fsanitize=undefined linkflags=-fno-sanitize-recover=undefined linkflags=-fuse-ld=gold variant=debug toolset=gcc-8 test_lock_concept
Performing configuration checks

    - default address-model    : 64-bit (cached)
    - default architecture     : x86 (cached)
    - symlinks supported       : yes (cached)
    - BOOST_COMP_GNUC >= 4.3.0 : yes (cached)
    - lockfree boost::atomic_flag : yes (cached)
...patience...
...patience...
...found 3533 targets...
...updating 2 targets...
testing.capture-output ../../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.run
====== BEGIN OUTPUT ======
../../../boost/test/impl/test_tree.ipp:273:25: runtime error: member call on address 0x7ffcf49a96f0 which does not point to an object of type 'test_unit_generator'
0x7ffcf49a96f0: note: object is of type 'boost::unit_test::ut_detail::template_test_case_gen<test_scoped_lock_concept_invoker, boost::mpl::vector<boost::mutex, boost::timed_mutex, boost::recursive_mutex, boost::recursive_timed_mutex, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >'
 11 56 00 00  68 1a 55 76 11 56 00 00  10 f8 9f 77 11 56 00 00  50 00 a0 77 11 56 00 00  04 00 00 00
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for 'boost::unit_test::ut_detail::template_test_case_gen<test_scoped_lock_concept_invoker, boost::mpl::vector<boost::mutex, boost::timed_mutex, boost::recursive_mutex, boost::recursive_timed_mutex, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na> >'

EXIT STATUS: 1
====== END OUTPUT ======

    LD_LIBRARY_PATH="/boost/bin.v2/libs/chrono/build/gcc-8/debug/threading-multi/visibility-hidden:/boost/bin.v2/libs/system/build/gcc-8/debug/threading-multi/visibility-hidden:/boost/bin.v2/libs/test/build/gcc-8/debug/threading-multi/visibility-hidden:/boost/bin.v2/libs/thread/build/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden:/boost/bin.v2/libs/timer/build/gcc-8/debug/threading-multi/visibility-hidden:/usr/bin:/usr/lib:/usr/lib32:/usr/lib64:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH

    status=0
    if test $status -ne 0 ; then
        echo Skipping test execution due to testing.execute=off
        exit 0
    fi
     "../../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept"   > "../../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output" 2>&1 < /dev/null
    status=$?
    echo >> "../../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output"
    echo EXIT STATUS: $status >> "../../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output"
    if test $status -eq 0 ; then
        cp "../../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output" "../../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.run"
    fi
    verbose=0
    if test $status -ne 0 ; then
        verbose=1
    fi
    if test $verbose -eq 1 ; then
        echo ====== BEGIN OUTPUT ======
        cat "../../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.output"
        echo ====== END OUTPUT ======
    fi
    exit $status

...failed testing.capture-output ../../../bin.v2/libs/thread/test/test_lock_concept.test/gcc-8/debug/threadapi-pthread/threading-multi/visibility-hidden/test_lock_concept.run...
...failed updating 1 target...

Some headers fail to compile independently

I think the previously-fixed Trac ticket #12228 has regressed.

The Boost Header Policy requires:

Make sure that a translation unit consisting of just the contents of the header file will compile successfully.

Yet, I'm seeing compile errors for quite a few of the headers under Clang and GCC. Eg for boost/test/data/monomorphic/initializer_list.hpp:

boost/test/data/monomorphic/initializer_list.hpp:45:27: error: no type named 'vector' in namespace 'std'
    typedef typename std::vector<T>::const_iterator iterator;
            ~~~~~~~~~~~~~~^~~~~~
boost/test/data/monomorphic/initializer_list.hpp:45:33: error: expected member name or ';' after declaration specifiers
    typedef typename std::vector<T>::const_iterator iterator;
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
boost/test/data/monomorphic/initializer_list.hpp:64:5: error: unknown type name 'iterator'
    iterator        begin() const   { return m_data.begin(); }
    ^
boost/test/data/monomorphic/initializer_list.hpp:68:10: error: no template named 'vector' in namespace 'std'
    std::vector<T> m_data;
    ~~~~~^
boost/test/data/monomorphic/initializer_list.hpp:79:18: error: no template named 'vector' in namespace 'std'
    typedef std::vector<bool>::const_iterator iterator;
            ~~~~~^
boost/test/data/monomorphic/initializer_list.hpp:79:32: error: non-friend class member 'const_iterator' cannot have a qualified name
    typedef std::vector<bool>::const_iterator iterator;
                             ~~^
boost/test/data/monomorphic/initializer_list.hpp:79:32: error: typedef declarator cannot be qualified
    typedef std::vector<bool>::const_iterator iterator;
                             ~~^
boost/test/data/monomorphic/initializer_list.hpp:79:46: error: expected ';' at end of declaration list
    typedef std::vector<bool>::const_iterator iterator;
                                             ^
                                             ;
boost/test/data/monomorphic/initializer_list.hpp:96:5: error: unknown type name 'iterator'
    iterator        begin() const   { return m_data.begin(); }
    ^
boost/test/data/monomorphic/initializer_list.hpp:100:10: error: no template named 'vector' in namespace 'std'
    std::vector<bool> m_data;
    ~~~~~^
10 errors generated.

I think there are problems compiling the following 20 headers:

boost/test/data/dataset.hpp
boost/test/data/monomorphic/delayed.hpp
boost/test/data/monomorphic/grid.hpp
boost/test/data/monomorphic.hpp
boost/test/data/monomorphic/initializer_list.hpp
boost/test/data/monomorphic/join.hpp
boost/test/data/monomorphic/zip.hpp
boost/test/data/test_case.hpp
boost/test/tools/collection_comparison_op.hpp
boost/test/tools/detail/lexicographic_manip.hpp
boost/test/tools/detail/per_element_manip.hpp
boost/test/tree/test_case_template.hpp
boost/test/utils/runtime/argument_factory.hpp
boost/test/utils/runtime/cla/argv_traverser.hpp
boost/test/utils/runtime/cla/parser.hpp
boost/test/utils/runtime/env/fetch.hpp
boost/test/utils/runtime/finalize.hpp
boost/test/utils/runtime/modifier.hpp
boost/test/utils/runtime/parameter.hpp
boost/test/utils/xml_printer.hpp

Thanks for your continued work on this library.

Significant start slowdown on MSVC x64/Debug after upgrade to v1.68.0

Hi!

I'm experiencing a very bad problem: I have a one boost test executable which contains over 50.000 test cases (data test cases mostly), and after upgrade to the latest Boost.Test the startup time has significantly increased from <3sec up to ~7min.

Boost.Test appears to crash under Cygwin

The tests for any Boost library that uses Boost.Test seem to crash for me under Cygwin with:

Starting program: /cygdrive/c/boost-git/develop/bin.v2/libs/range/test/std_conta
iner.test/gcc-6.4.0/debug/link-static/target-os-cygwin/threading-multi/std_conta
iner.exe
[New Thread 56732.0xb798]
[New Thread 56732.0x8d3c]
[New Thread 56732.0xb228]
[New Thread 56732.0xd984]

Program received signal SIGSEGV, Segmentation fault.
0x004238ab in boost::unit_test::framework::get (id=2147824840,
    t=boost::unit_test::TUT_CASE) at ../boost/test/impl/framework.ipp:1490
1490        BOOST_TEST_I_ASSRT( (res->p_type & t) != 0, internal_error( "Invalid
 test unit type" ) );
(gdb) bt
#0  0x004238ab in boost::unit_test::framework::get (id=2147824840,
    t=boost::unit_test::TUT_CASE) at ../boost/test/impl/framework.ipp:1490
#1  0x0044611d in boost::unit_test::framework::get<boost::unit_test::test_case>
(id=2147824840) at ../boost/test/framework.hpp:232
#2  0x00405dbd in boost::unit_test::traverse_test_tree (id=2147824840, V=...,
    ignore_status=true) at ../boost/test/impl/test_tree.ipp:363
#3  0x00423041 in boost::unit_test::framework::finalize_setup_phase (
    master_tu_id=2147824840) at ../boost/test/impl/framework.ipp:1163
#4  0x0040b067 in boost::unit_test::unit_test_main (init_func=
    0x401ef6 <init_unit_test_suite(int, char**)>, argc=1, argv=0x28cc0c)
    at ../boost/test/impl/unit_test_main.ipp:200
#5  0x0040b4a0 in main (argc=1, argv=0x28cc0c)
    at ../boost/test/impl/unit_test_main.ipp:286

This example is for Range's std_container.cpp.

Feature Request: BOOST_TEST_CONTEXT that doesn't require introducing a new scope with braces

I've been using Boost.Test for a project at work, and one thing I found annoying is that BOOST_TEST_CONTEXT requires you to create a new scope using braces, and the context information will only live as long as that new scope. What I originally though it would do (and what I really want it to do) is to make the additional context information last for the remainder of the enclosing scope. This allows me to slowly build context information as my test progresses without requiring excessive indentation.

My current workaround is to use a define as below (notice the declaration is no longer in an "if" condition):

#define MY_BOOST_TEST_CONTEXT( context_descr )                                                  \
    ::boost::test_tools::tt_detail::context_frame BOOST_JOIN( context_frame_, __LINE__ ) =      \
        ::boost::test_tools::tt_detail::context_frame( BOOST_TEST_LAZY_MSG( context_descr ) )

This allows me to do this:

for (int i = 8; i < 24; ++i) {
    MY_BOOST_TEST_CONTEXT("corrupting CRC bit " << i);
    // ...
}

Instead of this:

for (int i = 8; i < 24; ++i) {
    BOOST_TEST_CONTEXT("corrupting CRC bit " << i) {
        // ...
    }
}

This particular example may not look like much, but after 3 or 4 additions to the context as my test progresses the indentation starts to get excessive.

empty test should pass successfuly

lets say we have single disabled test:
BOOST_AUTO_TEST_CASE(test_name ,*disabled())

it should not crash/fail:
boost::unit_test::ut_detail::throw_exceptionboost::unit_test::framework::setup_error(e=) at throw_exception.hpp:48 [opt]
45
46 template
47 BOOST_NORETURN inline void
-> 48 throw_exception(E const& e) { �[4mt�[0mhrow e; }
49
50 #define BOOST_TEST_I_TRY try
51 #define BOOST_TEST_I_CATCH( T, var ) catch( T const& var )

instead it may successfully end with 0 tests passed

	The third form is an extension of the previous form for datasets of arity `N`. The sample being a polymorphic tuple, each
	of the variables `var1`, ..., `varN` corresponds to the index 1, ... `N` of the the sample:

	[snippet_dataset1_3]

	//[snippet_dataset1_3
	BOOST_DATA_TEST_CASE( test_case_arity2, data::xrange(2) ^ data::xrange(5), apples, potatoes)
	{
	BOOST_TEST((apples <= 1 && apples >= 0));
	BOOST_TEST((potatoes <= 4 && potatoes >= 0));
	}
	//]

	A particular handling is performed if `dsa` and `dsb` are of different size. The rule is as follow:

	* if the both zipped datasets have the same size, this is the size of the resulting dataset (this size can then be infinite).
	* otherwise if one of the dataset is of size 1 (singleton) or of infinite size, the resulting size is governed by the other dataset.
	* otherwise an exception is thrown at runtime

	m_checkpoint_data.m_file_name = file;
	m_checkpoint_data.m_line_num = line_num;