sheredom / utest.h Goto Github PK
View Code? Open in Web Editor NEW🧪 single header unit testing framework for C and C++
License: The Unlicense
utest.h's People
Contributors
Stargazers
Watchers
Forkers
ldrumm squidfarts khiemhertz ubuntu-repo c-and-cpp-libraries szaydel fulg skyformat99 yangxingpping nikishyn mxe191 peterlindgren haifenghuang friendlyanon akkaze jatobu mirko796 tjachmann ibrahim-786 qrikko aganm bwuck yesudeep iomeone philck draperdanman tekknolagi watch-later ajunlonglive yishengyounitt mohabouje ibutra frithrah bravegnu timgates42 alecrothenberg boribs beered rasky gmh5225 ericrobolson akafael asmibug xqyjlj waywardmonkeys iknowmagic bluehazzard ksil ericonr gottox pollend georges987 peyman-esteghamatrad oss-cosmicutest.h's Issues
Any way to deploy test statements in function callback?
ASSERT_NE shows incorrect text
Using ASSERT_NE(0, 0) will report an failure like this
file.c:14: Failure
Expected : 0
Actual : 0
The text "Expected" here isn't correct.
msys2 + windows + gcc: #error Unsupported platform! 302 | #error Unsupported platform!
Error:
../../../third_party/utest/utest.h: In function 'utest_ns':
../../../third_party/utest/utest.h:302:2: error: #error Unsupported platform!
302 | #error Unsupported platform!
| ^~~~~
compilation terminated due to -Wfatal-errors.
> uname -sr
MSYS_NT-10.0-22000 3.2.0-340.x86_64
> gcc --version
gcc.exe (Rev1, Built by MSYS2 project) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Possible solution:
- Check for the presence of
__MINGW64__and__MINGW32__preprocessor definitions alongside_MSC_VER.
error: conversion from ‘size_t’ compiling with -Werror=conversion
Thanks for this handy unit test framework! I use it in my mpdclient project myMPD.
Version: master
Toolchain: gcc version 11.2.0 (Ubuntu 11.2.0-19ubuntu1)
Target: x86_64-linux-gnu
Full error:
/dist/utest/utest.h: In function ‘utest_main’:
/dist/utest/utest.h:1212:35: error: conversion from ‘size_t’ {aka ‘long unsigned int’} to ‘utest_uint32_t’ {aka ‘const unsigned int’} may change value [-Werror=conversion]
1212 | const utest_uint32_t next = ((word >> 22u) ^ word) % index;
warning: non-void function does not return a value
Assuming one wants to compile utest.h cleanly without warnings, here's a warning report I've noticed:
❯ bazel test ...
INFO: From Compiling external/sheredom_utest/decoy_main.c:
In file included from bazel-out/freebsd-fastbuild/bin/external/sheredom_utest/decoy_main.c:1:
external/sheredom_utest/utest.h:266:1: warning: non-void function does not return a value [-Wreturn-type]
}
^
1 warning generated.
Sysinfo:
❯ clang --version
clang version 14.0.0
Target: x86_64-portbld-freebsd13.0
Thread model: posix
InstalledDir: /usr/local/llvm-devel/bin
few little changes without pull requests
Great TU lib, thanks :)
utest.h from approx line 848
#ifdef _MSC_VER
// DBJ: switch on VT100
// horrible but works on WIN10
// on other versions has no visible effect
system(" ");
#endif // _MSC_VER
const char **colours = make_colours() ;And from approx line 178
#ifdef _MSC_VER
#include <sdkddkver.h>
/*
io.h contains definitions for some structures with natural padding. This is
uninteresting, but for some reason MSVC's behaviour is to warn about
including this system header. That *is* interesting
*/
#pragma warning(disable : 4820)
#pragma warning(push, 1)
#include <io.h>
#pragma warning(pop)
#define UTEST_COLOUR_OUTPUT() ( (_isatty(_fileno(stdout))) && (_WIN32_WINNT >= _WIN32_WINNT_WIN10) )
// #endif
#else // ! _MSC_VER
#include <unistd.h>
#define UTEST_COLOUR_OUTPUT() (isatty(STDOUT_FILENO))
#endif // ! _MSC_VER
enum colours_enum { RESET, GREEN, RED };
static const char ** make_colours () {
static const char * colours_on[] = {"\033[0m", "\033[32m", "\033[31m"} ;
static const char *colours_off[] = {"", "", ""};
static const char **colours_ = NULL ;
if ( NULL == colours_ ) {
if (! UTEST_COLOUR_OUTPUT() ) {
colours_ = colours_off ;
} else {
colours_ = colours_on ;
}
}
return colours_ ;
}I added checking for WIN10 as there are no VT100 colours before that while using cmd.exe. I assume no sane person will use this from inside Win App so there is no point checking on that.
If one is using VS Code, (_isatty(_fileno(stdout)) will not work from inside VS Code debug terminal. In launch.json do the "externalConsole": true to rectify that.
HTH
error: pointer of type 'void *' used in arithmetic [-Werror=pointer-arith]
GCC complains about this when the "-Werror=pointer-arith" flag is turned on
and one uses EXPECT_EQ to compare pointers:
external/com_github_sheredom_utest/utest.h:528:36: error: pointer of type 'void *' used in arithmetic [-Werror=pointer-arith]
528 | #define UTEST_AUTO(x) __typeof__(x + 0)
| ^
external/com_github_sheredom_utest/utest.h:580:5: note: in expansion of macro 'UTEST_AUTO'
580 | UTEST_AUTO(y) yEval = (y); \
| ^~~~~~~~~~
external/com_github_sheredom_utest/utest.h:633:25: note: in expansion of macro 'UTEST_EXPECT'
633 | #define EXPECT_EQ(x, y) UTEST_EXPECT(x, y, ==)
| ^~~~~~~~~~~~
foo/foo_test.c:42:3: note: in expansion of macro 'EXPECT_EQ'
42 | EXPECT_EQ(p, NULL);
GCC version:
>> gcc --version
gcc (Clear Linux OS for Intel Architecture) 12.1.1 20220720 releases/gcc-12.1.0-247-g94d44a83ff
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
OS:
>> uname -mrs
Linux 5.18.12-1160.native x86_64
I can work around this by using EXPECT_TRUE to make pointer comparisons, but wonder whether there is a fix for this.
Allow only compiling implementation details once
Would it be possible to support a macro like UTEST_IMPLEMENTATION, similar to how it's done e.g. in the stb libs like stb_image.h?
In utest.h, the declarations and macros would come at the top, and then an #ifdef UTEST_IMPLEMENTATION. And after that, all the implementations and internal details.
So as a user, when you normally #include "utest.h", you only get the declarations and macros. And then, in exactly one .c/.cpp file, you only have:
#define UTEST_IMPLEMENTATION
#include "utest.h"
This time, because UTEST_IMPLEMENTATION is defined, utest.h is compiled with all the implementations.
This avoids recompiling most of utest.h over and over while you're writing/changing your test cases. I hope I explained this in a way that's not confusing. And thank you for making utest 🙂
I would be willing to try this and make a PR, but first wanted to ask if you're interested, because the changes are probably not very diff-friendly 😅
warning: unused parameter 'utest_result'
Could this please be addressed as well?
In file included from lab/bugs/use_after_free_asan_test.c:4:
lab/bugs/use_after_free_asan_test.c: In function 'utest_run_UseAfterFreeTest_WillFailWhenAsanIsEnabled':
external/sheredom_utest/utest.h:764:38: error: unused parameter 'utest_result' [-Werror=unused-parameter]
764 | void utest_run_##SET##_##NAME(int *utest_result)
| ~~~~~^~~~~~~~~~~~
lab/bugs/use_after_free_asan_test.c:6:1: note: in expansion of macro 'UTEST'
6 | UTEST(UseAfterFreeTest, WillFailWhenAsanIsEnabled) {
| ^~~~~
cc1: all warnings being treated as errors
INFO: Elapsed time: 0.250s, Critical Path: 0.06s
INFO: 13 processes: 13 internal.
FAILED: Build did NOT complete successfully
Thank you for writing an awesome library!
NAN makes test pass
I have this issue where a NaN makes my test pass:
UTEST(test, test)
{
float test_value = NAN;
float epsilon = 0.001f;
EXPECT_NEAR(1.f, test_value, epsilon); // test passes
}
Should I check for NaNs everytime I pass a number to a utest macro? Would it make sense for utest to check internally whether values passed to it are NaN before checking the condition, and failing the test if any of the values are NaN?
test is not run (VS15, Release)
struct utest_state is empty when run test in Release mode (Visual studio 15).
I have printed:
[==========] Running 0 test cases.
[==========] 0 test cases ran.
[ PASSED ] 0 tests.
Is it ok?
In Debug - all tests are running
Would you be interested in a literate program variant of utest.h?
Namaste,
I've outlined utest.h into an org-mode literate program for personal use, but I'm wondering whether you'd be interested in having utest.h generated by a literate program. Ideally, I wouldn't want to fork it and would prefer it be part of this repository. However, I'm not sure whether you'd be comfortable with Emacs+Org-Mode.
The literate program makes it easier to organize and manage the header file if you can deal with Emacs+Org-Mode. Please watch https://www.youtube.com/watch?v=wNwBbG5ow6g and let me know.
Thank you!
utest.h does not include time.h when compiling on ARM64 Manjaro Linux with Clang
Problem: When compiling the unit tests for my project, the Clang build fails with the following errors:
In file included from ./main.c:11:
././utest.h:200:19: error: variable has incomplete type 'struct timespec'
struct timespec ts;
^
././utest.h:200:10: note: forward declaration of 'struct timespec'
struct timespec ts;
^
././utest.h:201:9: error: unknown type name 'clockid_t'
const clockid_t cid = CLOCK_REALTIME;
^
././utest.h:201:25: error: use of undeclared identifier 'CLOCK_REALTIME'
const clockid_t cid = CLOCK_REALTIME;
^
././utest.h:203:3: warning: implicit declaration of function 'clock_gettime' is invalid in C99 [-Wimplicit-function-declaration]
clock_gettime(cid, &ts);
^
1 warning and 3 errors generated.
From some experimentation, I've narrowed the source of the problem down to time.h not being included due to __GLIBC__ and __GLIBC_MINOR__ becoming undefined somewhere between including C Standard Library headers like stdio.h and utest.h being included. I unfortunately do not have the experience to know exactly why, nor could I find an immediately obvious reason as to the cause in utest.h itself.
I have not reproduced this on other architectures using Linux, nor with other versions of Linux, but I do know that my code compiled without problem on Windows running on a 64-bit Intel processor.
Platform: aarch64 Linux (Manjaro ARM)
Compiler: clang 10.0.1
Steps to reproduce:
- Pull latest
utest.h - Create a source file
example.cwith the following contents:
#include <stdio.h>
#include "./utest.h"
UTEST_MAIN();- Compile
example.cwith using the following command:
clang -g -std=c99 ./example.cAny thoughts on why this could be happening? I plan to see if I can replicate this on Linux running on a 64-bit Intel processor to rule out the possibility of it being a problem with the POSIX headers used by Manjaro ARM.
Handle temporary strings in ASSERT_STREQ etc
Consider such code
#include "utest.h"
#include <string>
UTEST(cpp_lifetime, std_strings) {
ASSERT_STREQ(std::string{"hello"}.c_str(), "hello");
}It causes warning (which should be treated as error!)
utest.h/test/lifetime.cpp:5:16: warning: object backing the pointer will be destroyed at the end of the full-expression [-Wdangling-gsl]
ASSERT_STREQ(std::string("hello").c_str(), "hello");
^~~~~~~~~~~~~~~~~~~~
utest.h/test/../utest.h:990:26: note: expanded from macro 'ASSERT_STREQ'
const char *xEval = (x); \
But passes.
Moreover,
struct PureString {
std::string s_;
PureString(const char* s) : s_(s) {}
void erase() { for (size_t i = 0, n = s_.size(); i != n; ++i) s_[i] = '#'; }
~PureString() { erase(); }
const char* c_str() const { return s_.c_str(); }
};
UTEST(cpp_lifetime, pure_strings) {
ASSERT_STREQ(PureString("hello").c_str(), "hello");
}causes no warnings at all.
And it fails.
Randomize tests order
Something that would be useful is to have tests running in a (optional) non-deterministic order. This allows catch a bunch of bugs such as:
- Tests by accident depend on data that has been generated by a previous test.
- Tests that runs after some tests happens depends on other tests data
Errors with GCC
One test fails:
[ RUN ] utest_cmdline.filter_with_list
/home/vagrant/code/utest.h/test/main.c:87: Failure
Expected : undef
Actual : undef
The type_printers set also outputs to the console, which is not the case on Windows with MSVC:
[ RUN ] type_printers.stdc
-2147483648-9223372036854775808-9223372036854775808429496729518446744073709551615184467440737095516150.0000000.0000000.000000[ OK ] type_printers.stdc (352203ns)
[ RUN ] type_printers.cpp
-2147483648-9223372036854775808-9223372036854775808429496729518446744073709551615184467440737095516150.0000000.0000000.000000[ OK ] type_printers.cpp (59254ns)
error: dereference of NULL '0' [CWE-476] [-Werror=analyzer-null-dereference]
Error:
../../../third_party/utest/utest.h: In function 'utest_main':
../../../third_party/utest/utest.h:1077:47: error: dereference of NULL '0' [CWE-476] [-Werror=analyzer-null-dereference]
1077 | failed_testcases[failed_testcase_index] = index;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~
compilation terminated due to -Wfatal-errors.
System information:
❯ uname -sr
FreeBSD 13.0-RELEASE
❯ gcc12 --version
gcc12 (FreeBSD Ports Collection) 12.0.0 20210919 (experimental)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
musl libc support
I'm using Alpine linux, a linux distribution based on musl libc instead of glibc, and i'm encountering a couple of compilation issues
First this issue about the time.h header that is not included
In file included from /home/simon/dev/utest.h/test/main.c:26:
/home/simon/dev/utest.h/test/../utest.h: In function ‘utest_ns’:
/home/simon/dev/utest.h/test/../utest.h:200:19: error: storage size of ‘ts’ isn’t known
200 | struct timespec ts;
| ^~
/home/simon/dev/utest.h/test/../utest.h:204:9: error: unknown type name ‘clockid_t’
204 | const clockid_t cid = CLOCK_REALTIME;
| ^~~~~~~~~
/home/simon/dev/utest.h/test/../utest.h:204:25: error: ‘CLOCK_REALTIME’ undeclared (first use in this function)
204 | const clockid_t cid = CLOCK_REALTIME;
| ^~~~~~~~~~~~~~
/home/simon/dev/utest.h/test/../utest.h:204:25: note: each undeclared identifier is reported only once for each function it appears in
/home/simon/dev/utest.h/test/../utest.h:208:11: error: ‘SYS_clock_gettime’ undeclared (first use in this function)
208 | syscall(SYS_clock_gettime, cid, &ts);
| ^~~~~~~~~~~~~~~~~
/home/simon/dev/utest.h/test/../utest.h:200:19: error: unused variable ‘ts’ [-Werror=unused-variable]
200 | struct timespec ts;
| ^~
/home/simon/dev/utest.h/test/../utest.h:215:1: error: control reaches end of non-void function [-Werror=return-type]
215 | }
| ^
Which is easily fixed by this patch
diff --git a/utest.h b/utest.h
index 7f2bcaa..5ade166 100644
--- a/utest.h
+++ b/utest.h
@@ -101,6 +101,10 @@ typedef uint64_t utest_uint64_t;
#include <sys/syscall.h>
#include <unistd.h>
#endif
+#else /* non glibc platforms */
+#include <time.h>
+#include <sys/syscall.h>
+#include <unistd.h>
#endif
#elif defined(__APPLE__)But then the real trouble begins
[ 50%] Linking CXX executable utest_test_wpo
/usr/bin/ld: cannot find -lgcc_
This caused by the fact that the project is using the gnu standard instead of the regular one. Changing it causes another wave of compilation issues, mostly due to comments and other non-standard things that gnu89 has.
[ 6%] Building C object CMakeFiles/utest_test_wpo.dir/main.c.o
/home/simon/dev/utest.h/test/main.c:1:1: error: C++ style comments are not allowed in ISO C90
1 | // This is free and unencumbered software released into the public domain.
| ^
/home/simon/dev/utest.h/test/main.c:1:1: note: (this will be reported only once per input file)
In file included from /home/simon/dev/utest.h/test/main.c:26:
/home/simon/dev/utest.h/test/../utest.h:160:22: error: unknown type name ‘inline’
160 | #define UTEST_INLINE inline
| ^~~~~~
/home/simon/dev/utest.h/test/../utest.h:195:8: note: in expansion of macro ‘UTEST_INLINE’
195 | static UTEST_INLINE utest_int64_t utest_ns(void) {
| ^~~~~~~~~~~~
/home/simon/dev/utest.h/test/../utest.h:195:35: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘utest_ns’
195 | static UTEST_INLINE utest_int64_t utest_ns(void) {
| ^~~~~~~~
/home/simon/dev/utest.h/test/../utest.h:798:21: error: expected ‘;’ before ‘int’
798 | static UTEST_INLINE int utest_strncmp(const char *a, const char *b, size_t n) {
| ^~~
/home/simon/dev/utest.h/test/../utest.h:160:22: error: unknown type name ‘inline’
160 | #define UTEST_INLINE inline
| ^~~~~~
/home/simon/dev/utest.h/test/../utest.h:813:8: note: in expansion of macro ‘UTEST_INLINE’
813 | static UTEST_INLINE FILE *utest_fopen(const char *filename, const char *mode) {
| ^~~~~~~~~~~~
/home/simon/dev/utest.h/test/../utest.h:813:26: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token
813 | static UTEST_INLINE FILE *utest_fopen(const char *filename, const char *mode) {
| ^
/home/simon/dev/utest.h/test/../utest.h: In function ‘utest_main’:
/home/simon/dev/utest.h/test/../utest.h:870:28: error: implicit declaration of function ‘utest_fopen’ [-Werror=implicit-function-declaration]
870 | utest_state.output = utest_fopen(argv[index] + strlen(output_str), "w+");
| ^~~~~~~~~~~
/home/simon/dev/utest.h/test/../utest.h:870:26: error: assignment to ‘FILE *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
870 | utest_state.output = utest_fopen(argv[index] + strlen(output_str), "w+");
| ^
/home/simon/dev/utest.h/test/../utest.h:917:10: error: implicit declaration of function ‘utest_ns’; did you mean ‘utest_main’? [-Werror=implicit-function-declaration]
917 | ns = utest_ns();
| ^~~~~~~~
| utest_main
In file included from /home/simon/dev/utest.h/test/main.c:27:
/home/simon/dev/utest.h/test/process.h: At top level:
/home/simon/dev/utest.h/test/process.h:156:3: error: C++ style comments are not allowed in ISO C90
156 | // stdout and stderr are the same FILE.
| ^
/home/simon/dev/utest.h/test/process.h:156:3: note: (this will be reported only once per input file)
/home/simon/dev/utest.h/test/process.h: In function ‘process_create’:
/home/simon/dev/utest.h/test/process.h:346:57: error: missing terminating ' character [-Werror]
346 | environment, // use parent's environment
| ^
/home/simon/dev/utest.h/test/process.h:347:57: error: missing terminating ' character [-Werror]
347 | NULL, // use parent's current directory
| ^
/home/simon/dev/utest.h/test/process.h:359:12: error: missing terminating ' character [-Werror]
359 | // We don't need the handle of the primary thread in the called process.
| ^
/home/simon/dev/utest.h/test/process.h:430:31: error: implicit declaration of function ‘fdopen’; did you mean ‘fopen’? [-Werror=implicit-function-declaration]
430 | out_process->stdin_file = fdopen(stdinfd[1], "wb");
| ^~~~~~
| fopen
/home/simon/dev/utest.h/test/process.h:430:29: error: assignment to ‘FILE *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
430 | out_process->stdin_file = fdopen(stdinfd[1], "wb");
| ^
/home/simon/dev/utest.h/test/process.h:435:30: error: assignment to ‘FILE *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
435 | out_process->stdout_file = fdopen(stdoutfd[0], "rb");
| ^
/home/simon/dev/utest.h/test/process.h:444:32: error: assignment to ‘FILE *’ from ‘int’ makes pointer from integer without a cast [-Werror=int-conversion]
444 | out_process->stderr_file = fdopen(stderrfd[0], "rb");
| ^
cc1: all warnings being treated as errors
I can take care of this, but I'd like to discuss about a strategy first. Would it be ok to bump the minimal required standard to c99?
Logging
Hi,
I'm trying to figure it out how the output of all tests are suppresed, so any of cout or printouts are not visible.
There is an output member inside utest_state_s but its only for printout for XML, how can I get the cout/printouts to be visible?
MSVC : C2733 Repeated linking of external C functions
C2733 : QueryPerformanceCounter & QueryPerformanceFrequency : Cannot overload a function with an external 'C'.
# include "utest.h"
# include <windows.h>
Env : win7 sp1, vs2019 /std=c++17
utest_ns() could not resolve the platform while using TDM-GCC-64
I hate it when people send me fixes to issue instead of PR .. but no time for much else than this :)
While using GCC (TDM-GCC-64) on WIN10 utest_ns() could not resolve the platform. This is the modified version:
basically I changed from checking _MSC_VER to _WIN32 when platform (not compiler) resolution is needed:
static UTEST_INLINE utest_int64_t utest_ns(void) {
#if defined(_WIN32) // DBJ: was _MSC_VER
utest_large_integer counter; // DBJ: now had to change arround line 74, see bellow
utest_large_integer frequency;
QueryPerformanceCounter(&counter);
QueryPerformanceFrequency(&frequency);
return UTEST_CAST(utest_int64_t,
(counter.QuadPart * 1000000000) / frequency.QuadPart);
#elif defined(__linux) && defined(__STRICT_ANSI__)
return UTEST_CAST(utest_int64_t, clock()) * 1000000000 / CLOCKS_PER_SEC;
#elif defined(__linux)
struct timespec ts;
#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L)
timespec_get(&ts, TIME_UTC);
#else
const clockid_t cid = CLOCK_REALTIME;
#if defined(UTEST_USE_CLOCKGETTIME)
clock_gettime(cid, &ts);
#else
syscall(SYS_clock_gettime, cid, &ts);
#endif
#endif
return UTEST_CAST(utest_int64_t, ts.tv_sec) * 1000 * 1000 * 1000 + ts.tv_nsec;
#elif __APPLE__
return UTEST_CAST(utest_int64_t, mach_absolute_time());
#elif __EMSCRIPTEN__
return emscripten_performance_now()*1000000.0;
#else
// DBJ: added
#error ------------------>> Couild not resolve the platform in utest_ns () <<------------------------
#endif
}And that slightly opened a can of worms, so I had to change this too:
// DBJ: arround line 74
#if defined(_WIN32) // DBJ: was _MSC_VER
// define UTEST_USE_OLD_QPC before #include "utest.h" to use old QueryPerformanceCounter
#ifndef UTEST_USE_OLD_QPC
#pragma warning(push, 0)
#include <Windows.h>
#pragma warning(pop)Those pragma warnings now produce warnings (sic) when GCC is used ...
Sorry, this is all I have time for...
warning C6308 in Visual Studio 2019
Hi,
The following warning is being reported in Microsoft Visual Studio Community 2019 (Version 16.6.5):
Warning C6308 'realloc' might return null pointer: assigning null pointer to 'utest_state.tests', which is passed as an argument to 'realloc', will cause the original memory block to be leaked.
The warning is reported for every test cases associated with an explicitly-defined fixture.
Can you help to resolve this warning? (For a cleaner build log)
Thanks.
P.S. We're using utest in MCUT (https://cutdigital.github.io/mcut.github.io/)
Fix compare macros #106 adds side effects, but lots of side effects
Some of my tests starting failing after #106 because #106 adds a few side effects in the macros that it changed, for example:
Lines 779 to 781 in 24e8f47
So I thought I was gonna fix the side effects in those few macros, but there are side effects all over the place in many more macros. It seems weird to only fix the side effects in these handful of macros when all the other side effects are left untouched.
I also noticed that a few macros already handle the side effects with UTEST_AUTO to evaluate the macro parameters only once, for example:
Lines 892 to 895 in 24e8f47
Should UTEST_AUTO be used in all UTEST_ASSERT/EXPECT macros so to fix all the side effects?
uninitialized index member of the utest_test_state_t structure
After line #644 the assignment to the index member is what might be mising?
utest_state.tests[index].func = &utest_##SET##_##NAME; \
utest_state.tests[index].name = name; \
utest_state.tests[index].index = index; \Test does not run from inside statically linked library (gcc)
I started organizing groups of related test files into their own static libraries such that all the static libraries will get linked together into a single test executable. The reason for this is because I need to compile multiple test executables which have different characteristics, which does not affect test code, but can affect the code under test. So instead of recompiling the tests everytime, I want to put the tests in a static library which will be reused for all of my test executables, cutting down my compile times by a significant amount.
The issue is that test cases that are inside a static library do not get run from the executable's UTEST_MAIN.
I tested Could it be something with UTEST global arrays not being shared across the static library and the executable?__attribute__((constructor)) from inside a static library and the attribute works as expected.
Edit: I think I tested the __attribute__((constructor)) wrong, I'm trying it again and it won't run anymore. I'm getting hella confused.
Edit 2: Okay.. I think I know what is going on here.
-
main executable + static library
__attribute__((constructor))function, the constructor does not run. -
main executable + static library
__attribute__((constructor))+ regular function,
when the main executable calls the regular function from the same file as the constructor in the static library,
the constructors inside the static library get called!
So the solution would seem to be: put a bootstrap function inside each test file in the static library, and call each bootstrap function from the executable to trigger the constructors in the static library.
Ooooff.
Color print on Visual Studio older than 2019
- Hi, when running tests I have printed:
�[32m[==========]�[0m Running 1 test cases.
�[32m[ RUN ]�[0m Test0.uint8Test_default
�[32m[ OK ]�[0m Test0.uint8Test_default (7700ns)
�[32m[==========]�[0m 1 test cases ran.
�[32m[ PASSED ]�[0m 1 tests.
Windows 10 Pro, Vers 1909, Build: 18363.1139
MSVC 2015.
- it seems on line 923 there is an attempt to read an uninitialized variable.
utest_state.tests[index].func(&result, utest_state.tests[index].index);
typeof not declared in C++ on GCC
If I try to use utest using any of the macros that internally use typeof for a c++ program using gcc 8.3.0, I get
utest.h:423:5: error: ‘typeof’ was not declared in this scope
A fix would be to use __typeof__ instead. I can fork and create a pull request if that is a valid fix and I am not missing anything.
I think the same is true for clang, not sure right now though.
Errors with Clang
The lack of do { /* statements... */ } while(0) in macros causes Clang to error out due to the -Wextra-semi-stmt flag.
unknown type name __auto_type
utest.h:396:23: error: unknown type name '__auto_type'
CentOS-7.5.1804 gcc.4.8.5
when I use typeof to replace it, it seems ok now.
error: extra ';' outside of a function [-Werror,-Wextra-semi] UTEST_MAIN();
When one uses a semicolon to terminate the macro call.
foo.c:112:13: error: extra ';' outside of a function [-Werror,-Wextra-semi]
UTEST_MAIN();
^
1 error generated.
error: Null Dereference pointer `utest_state.tests`
Found 15 (out of 15) source files to analyze in ~/code/foo/infer-out
15/15 [##################################################################] 100% 1.856s
lab/bugs/leakMemoryAsan_test.c:6: error: Null Dereference
pointer `utest_state.tests` last assigned on line 6 could be null and is dereferenced at line 6, column 1.
4. #include "utest.h"
5.
6. UTEST(LeakMemoryTest, DetectedWithLsanOrMiMalloc_1) {
^
Array testing macro
Is there an easy way to compare two arrays?
Proposal to add TODO test result with yellow color tag
Why?
When writing tests it is common to put some unimportant function aside due to low-priority in order to implement tests later. It is a very dangerous practice, because you might think that the whole module is tested, but it is not always the case and some functions might be left out for later and eventually forgotten. It would be really nice if framework would allow to call UTEST_TODO() to mark test as unimplemented which would show it as yellow in the summary.
One can argue you should use test coverage utils but lets be honest it is rarely the case.
How?
Add another result OK, FAILED, TODO with yellow color marker.
Who?
I can do it if you are willing to assist along the way to eventually PR it (so no-ones time is wasted). But if it is an easy feature I would rather let other people familiar with the library do it.
Unrelated Notes
Very nice library. The fact that tests are auto-added is a game changer. Have a great day!
Add support for asserts with epsilon
When testing if a float is the desired value it's often a good idea to include the test within a epsilon due to the nature of floating points so the value gets tested +/- within that range (such as 0.0001f)
work like doctest for c++?
doctest has one definition flag(DOCTEST_CONFIG_DISABLE) that can toggle testcase code(e.g. like NDEBUG) and can let me mix test code with production code easily, can utest.h do that? that is, all unit tests code are right in the production code(instead of in separate test source files), and I can turn them off for a release build.
that make all test code just like the NDEBUG controlled assert, I can turn on and off easily, and can put testcases with the real production code together, in the same file.
however doctest does not work for c, utest.h can, so I wonder if utest.h can do the same.
utest.h doesn't compile in WSL
Problem
utest.h won't compile on WSL.
It seems to be related to the time function.
> make
gcc -Isrc/ -std=c11 -W -Wall -Wextra -pedantic -Os -g3 -Wshadow -Wdouble-promotion -Wformat=2 -Wformat-truncation -Wundef -fno-common -Wconversion -fsanitize=address -c tests/tests.c
In file included from tests/tests.c:5:
tests/utest.h: In function ‘utest_ns’:
tests/utest.h:201:9: error: unknown type name ‘clockid_t’
201 | const clockid_t cid = CLOCK_REALTIME;
| ^~~~~~~~~
tests/utest.h:201:25: error: ‘CLOCK_REALTIME’ undeclared (first use in this function)
201 | const clockid_t cid = CLOCK_REALTIME;
| ^~~~~~~~~~~~~~
tests/utest.h:201:25: note: each undeclared identifier is reported only once for each function it appears in
tests/utest.h:203:3: warning: implicit declaration of function ‘clock_gettime’ [-Wimplicit-function-declaration]
203 | clock_gettime(cid, &ts);
| ^~~~~~~~~~~~~
Makefile:32: recipe for target 'tests.o' failed
make: *** [tests.o] Error 1More info
OS: Windows10 x64
WSL OS: ubuntu 18
Windows Build: 18363.900
Compiler: gcc (Ubuntu 9.3.0-11ubuntu0~18.04.1) 9.3.0
GLIBC: ldd (Ubuntu GLIBC 2.27-3ubuntu1.2) 2.27
Regards
Manage exceptions with specific message inside
I've seen this problem in Google Test (google/googletest#952), which doesn't provide a function for this. Some users have "solve" it by adding custom macros, like EXPECT_THROW_WITH_MESSAGE (can be seen in the same issue). Maybe something like that could be added to utest.h.
STRN*-checks assume null-terminated arguments
Hi,
Thanks for this excellent library, I'm using it for most of my tests now and it's been working great.
While browsing the header I noticed that the STRN*-checks all assume both arguments are null-terminated. I'd like to suggest a third argument, len, be added to let the user specify the desired string length to compare. My particular use case is that I'm writing tests for a lexer/parser where tokens point to the middle of an input string buffer.
So my suggestion is to change from this:
#define ASSERT_STRNEQ(x, y) \
if (0 != UTEST_STRNCMP(x, y, strlen(x))) { \
UTEST_PRINTF("%s:%u: Failure\n", __FILE__, __LINE__); \
UTEST_PRINTF(" Expected : \"%s\"\n", x); \
UTEST_PRINTF(" Actual : \"%s\"\n", y); \
*utest_result = 1; \
return; \
}
To this:
#define ASSERT_STRNEQ(x, y, len) \
if (0 != UTEST_STRNCMP(x, y, len)) { \
UTEST_PRINTF("%s:%u: Failure\n", __FILE__, __LINE__); \
UTEST_PRINTF(" Expected : \"%.*s\"\n", len, x); \
UTEST_PRINTF(" Actual : \"%.*s\"\n", len, y); \
*utest_result = 1; \
return; \
}
Best,
Peter
error: double-'free' of '<unknown>' [CWE-415] [-Werror=analyzer-double-free]
Error:
third_party/utest/utest.h: In function 'utest_main':
third_party/utest/utest.h:1122:5: error: double-'free' of '<unknown>' [CWE-415] [-Werror=analyzer-double-free]
1122 | free(UTEST_PTR_CAST(void *, utest_state.tests[index].name));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated due to -Wfatal-errors.
cc1: all warnings being treated as error
Build configuration:
❯ ./configure
...
Configuration parameters:
AR_FLAGS cru
CC gcc
CC vendor gnu
CC version 10.3.0
CFLAGS
-g
-O2
-std=c11
-fanalyzer
-fdiagnostics-color=always
-Wall
-Wextra
-Werror
-Wfatal-errors
-Wpedantic
-pedantic-errors
-Werror=missing-declarations
-Werror=unused
-Werror=unused-result
-Werror=unused-local-typedefs
-Werror=write-strings
-Werror=sign-conversion
-Werror=narrowing
-Werror=uninitialized
-Wformat=2
-Wformat-security
-Wformat-y2k
-fstack-protector-all
CPP gcc -E
CPPFLAGS
-U_FORTIFY_SOURCE
-D_FORTIFY_SOURCE=2
CXX g++
CXX vendor
CXX version
CXXFLAGS
-g
-O2
LDFLAGS
HOST
HOSTNAME
OSTYPE
uname -mrs FreeBSD 13.0-RELEASE amd64
host amd64-unknown-freebsd13.0
enable-hardening yes
enable-asan no
enable-tsan no
enable-ubsan no
Add support for custom messages
The idea is to add something similar to Google Test << operator, but maybe simpler. Like overloading macros like: ASSERT_TRUE(false); with ASSERT_TRUE(false, "custom message");, or maybe simply adding new functions like ASSERT_TRUE_MSG(false, "custom message");. I didn't see any issues discussing this, so I wanted to see if this has been avoided for some reason.
Is utest.h C99 compliant?
I noticed that it compiles fine with gcc -std=c99 -O3 -Wall but the ASSERT_EQ macro just displays 'undef' when the actual and expected things differ.
NULL check with clang-14
I'm using utest to test a my C library. In these tests I also have some C++ compatibility tests. Unfortunately it seems that utest in conjunction with clang-14 has issues here.
Reduced testcase:
Input source:
#include "utest.h"
UTEST(group, test) {
ASSERT_EQ(nullptr, nullptr);
}
UTEST_MAIN()clang 14 works fine in C (after replacing nullptr with NULL), but with clang++-14, I get the following error:
user934391@FF239 utest.h % clang++ main.cpp
main.cpp:4:2: error: invalid operands to binary expression ('std::nullptr_t' and 'int')
ASSERT_EQ(nullptr, nullptr);
^~~~~~~~~~~~~~~~~~~~~~~~~~~
./utest.h:800:25: note: expanded from macro 'ASSERT_EQ'
#define ASSERT_EQ(x, y) UTEST_COND(x, y, ==, "", 1)
^~~~~~~~~~~~~~~~~~~~~~~~~~~
./utest.h:724:21: note: expanded from macro 'UTEST_COND'
UTEST_AUTO(x) xEval = (x); \
^~~~~~~~~~~~~
./utest.h:682:32: note: expanded from macro 'UTEST_AUTO'
#define UTEST_AUTO(x) typeof(x + 0)
~ ^ ~
main.cpp:4:2: error: invalid operands to binary expression ('std::nullptr_t' and 'int')
ASSERT_EQ(nullptr, nullptr);
^~~~~~~~~~~~~~~~~~~~~~~~~~~
./utest.h:800:25: note: expanded from macro 'ASSERT_EQ'
#define ASSERT_EQ(x, y) UTEST_COND(x, y, ==, "", 1)
^~~~~~~~~~~~~~~~~~~~~~~~~~~
./utest.h:725:5: note: expanded from macro 'UTEST_COND'
UTEST_AUTO(y) yEval = (y); \
^~~~~~~~~~~~~
./utest.h:682:32: note: expanded from macro 'UTEST_AUTO'
#define UTEST_AUTO(x) typeof(x + 0)
~ ^ ~
2 errors generated.
user934391@FF239 utest.h %
The original testcase looks like this, but produces the same error
#ifndef __cplusplus
# error "This file is C++ only"
#endif
#include "common.h"
#include <sqsh_archive.h>
#include <sqsh_directory.h>
#include <sqsh_error.h>
#include <sqsh_mapper.h>
#include <utest.h>
UTEST(cpp_test, sqsh_empty) {
int rv;
struct SqshArchive *archive = NULL;
struct SqshConfig config = {};
config.source_mapper = sqsh_mapper_impl_static;
config.mapper_block_size = 1;
config.source_size = 0;
archive = sqsh_archive_open(NULL, &config, &rv);
ASSERT_EQ(-SQSH_ERROR_SUPERBLOCK_TOO_SMALL, rv);
ASSERT_EQ(NULL, archive);
}
UTEST_MAIN()opaque types and ASSERT_EQ
I'm facing the issue, that I have problems when I do NULL checks for opaque types:
#include "utest.h"
struct Foo;
UTEST(a, b) {
struct Foo *foo = NULL;
ASSERT_EQ(NULL, foo);
}
UTEST_MAIN();clang-17 works here, but if I compile with gcc-13 I get the following error:
In file included from t.c:1:
t.c: In function 'utest_run_a_b':
utest.h:678:36: error: invalid use of undefined type 'struct Foo'
678 | #define UTEST_AUTO(x) __typeof__(x + 0)
| ^
utest.h:754:5: note: in expansion of macro 'UTEST_AUTO'
754 | UTEST_AUTO(y) yEval = (y); \
| ^~~~~~~~~~
utest.h:800:25: note: in expansion of macro 'UTEST_COND'
800 | #define ASSERT_EQ(x, y) UTEST_COND(x, y, ==, "", 1)
| ^~~~~~~~~~
t.c:7:9: note: in expansion of macro 'ASSERT_EQ'
7 | ASSERT_EQ(NULL, foo);
| ^~~~~~~~~
utest.h:643:41: error: arithmetic on pointer to an incomplete type
643 | : _Generic((val - val), ptrdiff_t \
| ^
utest.h:405:33: note: in definition of macro 'UTEST_PRINTF'
405 | fprintf(utest_state.output, __VA_ARGS__); \
| ^~~~~~~~~~~
utest.h:765:7: note: in expansion of macro 'utest_type_printer'
765 | utest_type_printer(yEval); \
| ^~~~~~~~~~~~~~~~~~
utest.h:800:25: note: in expansion of macro 'UTEST_COND'
800 | #define ASSERT_EQ(x, y) UTEST_COND(x, y, ==, "", 1)
| ^~~~~~~~~~
t.c:7:9: note: in expansion of macro 'ASSERT_EQ'
7 | ASSERT_EQ(NULL, foo);
| ^~~~~~~~~
utest.h:643:41: error: arithmetic on pointer to an incomplete type
643 | : _Generic((val - val), ptrdiff_t \
| ^
utest.h:407:10: note: in definition of macro 'UTEST_PRINTF'
407 | printf(__VA_ARGS__)
| ^~~~~~~~~~~
utest.h:765:7: note: in expansion of macro 'utest_type_printer'
765 | utest_type_printer(yEval); \
| ^~~~~~~~~~~~~~~~~~
utest.h:800:25: note: in expansion of macro 'UTEST_COND'
800 | #define ASSERT_EQ(x, y) UTEST_COND(x, y, ==, "", 1)
| ^~~~~~~~~~
t.c:7:9: note: in expansion of macro 'ASSERT_EQ'
7 | ASSERT_EQ(NULL, foo);
| ^~~~~~~~~
Out of curiousity: Why is typeof(x + 0) instead of typeof(x) used here? - It looks like some compiler quirk that you wanted to workaround
Is there any use for a macro comparing raw bytes?
I was wondering why is there not a macro that uses compares raw bytes with memcmp() in utest.h?
I feel like it could be useful when you need to compare anything else other than strings. Of course you always can wrap memcmp() around ASSERT_TRUE but then you don't get the nice output that tells you at which offset there is a difference in case of failure.
Perhaps there is another way to do it that I missed?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.