bic-mni / libminc Goto Github PK
View Code? Open in Web Editor NEWlibminc is the core library and API of the MINC toolkit
License: Other
libminc is the core library and API of the MINC toolkit
License: Other
MINC - Medical Image NetCDF or MINC isn't netCDF INTRODUCTION ------------ The MINC file format is a highly flexible medical image file format built on the HDF5 generalized data format. The format is simple, self-describing, extensible, portable and N-dimensional, with programming interfaces for both low-level data access and high-level volume manipulation. On top of the libraries is a suite of generic image-file manipulation tools. The format, libraries and tools are designed for use in a medical-imaging research environment : they are simple and powerful and make no attempt to provide a pretty interface to users. Minc tools are hosted in https://github.com/BIC-MNI/minc-tools Additional tools are available from the Montreal Neurological Institute (MNI). https://packages.bic.mni.mcgill.ca/tgz SUPPORT FOR HDF5 "MINC 2.0" FORMAT FILES ---------------------------------------- This change requires that HDF5 must be installed before MINC can be built. You can obtain HDF5 from, current version is 1.8.11: https://www.hdfgroup.org/ You should NOT need to become an HDF5 expert to use MINC 2.0. However, two tools included with HDF5 may prove useful, "h5dump" and "h5ls". h5dump is roughly equivalent to netCDF's ncdump utility. There is no exact netCDF tool analogous to h5ls. h5ls is useful for exploring and extracting bits of the HDF5 hierarchy. 3. TENTATIVE ".mincrc" AND ENVIRONMENT VARIABLE SUPPORT This is implemented as a way to control certain behaviors of the library which are not readily available through other means. These variables may be defined either in your environment, or in the .mincrc file in your home directory. The value in the environment should override the one in .mincrc. Here's what exists so far: # Force output files to MINC 2.0 format, regardless of the "-2" option. # MINC_FORCE_V2 = {1, 0} # Desired ZLIB compression level. Zero implies no compression, a value # of 4 gives a good tradeoff of compression and performance. # MINC_COMPRESS = {0..9} # Desired HDF5 chunking dimension. This controls the size of the # "hypercube" used by HDF5 to store the file. If set to zero, the # file will not be stored in a chunked format, unless compression is # enabled. The chunk size will automatically be reduced if it exceeds # any actual dimension of the volume. If chunking is enabled # automatically because of compression, the default chunk dimension is # 32. # MINC_CHUNKING = {0..N} # Log file - path to the desirned output file for messages. The default # is the standard error (stderr) stream. To redirect to standard output, # set this variable to "stdout" or "-". Otherwise the variable is taken # to be the path to the desired log file. If the path name begins with # a '+' character, the log file should be appended rather than recreated. # MINC_LOGFILE = [+]<pathname>|stdout|- # Logging level. Not really useful yet, but intended to allow setting # of 'verbosity' of messages. A value of zero would inhibit all but # fatal messages, a value of 4 would allow debugging messages. # MINC_LOGLEVEL = 0-4 # Maximum memory buffer, used in some minc programs # MINC_MAX_FILE_BUFFER_KB = <N> # Maximum allocated memory, used in some minc programs # MINC_MAX_MEMORY_KB = <N> # Memory cache for MINC2 files, per open file # MINC_FILE_CACHE_MB = <N> # Creation of HDF5 checksums for the volume, only works for MINC2 # file format # default 0 # MINC_CHECKSUM = {1,0} # Prefer using MINC2 API inside Volume IO # default 0 # MINC_PREFER_V2_API = {1,0} DOCUMENTATION ------------- https://en.wikibooks.org/wiki/MINC
Hi,
The subject says it all :)
There are many place into libminc source code where type int is used in place of hid_t. It badly breaks with HDF5 1.10 where hid_t was promoted to 64 bits integer.
Thanks in advance.
Documentation says:
miget_dimension_sampling_flag returns True if dimension is regularly sampled, however https://github.com/BIC-MNI/libminc/blob/develop/libsrc2/dimension.c#L1111 returns True if dimension is irregularly sampled
same goes for https://github.com/BIC-MNI/libminc/blob/develop/libsrc2/dimension.c#L1124
Hello, the test minc2-dimension-test fails with
Error reported on line #297, wrong result: 0
Error reported on line #294, failed: -1
Error reported on line #297, wrong result: 1
Error reported on line #294, failed: -1
Error reported on line #297, wrong result: 2
Error reported on line #294, failed: -1
Error reported on line #297, wrong result: 3
...
37 errors reported
on powerpc. This is probably a big/low endian conversion problem.
This needs some kind of version checking, newer versions of microsoft compilers supply rint.
https://github.com/BIC-MNI/libminc/blob/develop/libsrc2/minc2_private.h#L210
Just a note that I'm increasingly getting people asking about using our various MINC atlases with tools such as ITK-SNAP and 3DSlicer https://github.com/Slicer
They seem to not be interested in touching minc-tools convert files.
Since these tools are built with ITK underneath, it shouldn't be "too hard" to enable MINC support at a build/interface level.
Currently this is implemented in CMake build with LIBMINC_INTEGRATE_VOLUME_IO option - maybe we should make it permanent?
Since quote a lot of changes to the function prototypes have been made, the documentation will probably have to be updated at https://en.wikibooks.org/wiki/MINC/Reference/MINC2.0_Application_Programmers_Interface
Consider hyper.c around line 1391:
free(temp_buffer);
clang analyzer correctly reports:
libminc/libsrc2/hyper.c:1391:5: Argument to free() is offset by 16 bytes from the start of memory allocated by malloc()
The APPLY_SCALING_NORM macro increments it's 3rd parameter, and thus by the time we get to free()ing it, it's not pointing to the base address. Evil.
The clock_gettime() API was (finally) added in macOS 10.12; it is not present in earlier versions.
libminc is checking that clock_gettime() is present on the machine that's doing the compiling, but that's no guarantee that it's present on the machine that runs the code. ex: if I build libminc on 10.13, but try to run the resulting executables on 10.10, they will crash.
There are a number of ways we could solve this.
https://gitlab.kitware.com/utils/kwsys/blob/master/kwsysPlatformTestsC.c
That way, HAVE_CLOCK_GETTIME will be false if the deployment target < 10.12.
People seem to think MINC is LGPL. I actually don't know.
Anyways, I don't see the LICENCE here.
If it is LGPL, it seems people think some some things about it that may not be true re: linking. We should proactively address these concerns.
Hello,
according to this discussion ITK contributed some fixes to minc, but apparently they are not yet included in the latest released version. Could you please make a new release with these new updates?
many thanks,
Gert
Start 45: minc2-label-test
45: Test command: /builddir/build/BUILD/libminc-libminc-2-3-00/build/testdir/minc2-label-test
45: Test timeout computed to be: 1500
45: /builddir/build/BUILD/libminc-libminc-2-3-00/libsrc2/label.c:120 (from MINC2): HDF5 function H5Tenum_nameof failed
45: /builddir/build/BUILD/libminc-libminc-2-3-00/libsrc2/label.c:148 (from MINC2): HDF5 function H5Tenum_valueof failed
45: Error reported on line #172, miget_label_value, White: -256
45: Error reported on line #175, miget_label_value, Blue: -256
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: Error reported on line #261, Voxel is not blue: 65280
45: Error reported on line #256, Voxel is not white: 0
45: 26 errors reported
45: Creating label image !!
45: 0
45: 6
45: Number of defined labels 6
45: Print label file with file order x,y,z
45: 0 ffffff00 80808000 ff00 ff0000 ff000000
45: 0 ffffff00 80808000 ff00 ff0000 ff000000
45: 0 ffffff00 80808000 ff00 ff0000 ff000000
45: 0 ffffff00 80808000 ff00 ff0000 ff000000
45: Print label file with apparent order z,y,x
45: 0 80808000 ff0000 80808000 ff0000 0
45: ff0000 0 80808000 0 80808000 ff0000
45: ffffff00 ff00 ff000000 ff00 ff000000 ffffff00
45: ff000000 ffffff00 ff00 ffffff00 ff00 ff000000
45/51 Test #45: minc2-label-test .................***Failed 0.01 sec
test_nan.mnc is a 10x10x10 voxel volume with regular values except for the following:
4,4,3 : Inf
4,4,4 : -Inf
4,4,5: NaN
Running following snippets:
minccalc -express 'isnan(A[0])' test_nan.mnc test_nan_nan.mnc
mincstats -sum test_nan_nan.mnc
Produces 2
Running following snippets:
mincreshape -short test_nan.mnc test_nan_short.mnc
minccalc -express 'isnan(A[0])' test_nan_short.mnc test_nan_short_nan.mnc
mincstats -sum test_nan_short_nan.mnc
Produces 0
minccalc relies on voxel_loop to work with the volume.
git master has a warning:
volume_io/Prog_utils/time.c:141:9: warning: implicit declaration of function 'gettimeofday' is invalid in C99 [-Wimplicit-function-declaration]
time.c starts with:
#if HAVE_SYS_TIME_H
# include <sys/time.h>
#else
# include <time.h>
#endif
I've confirmed that I don't get into "#if HAVE_SYS_TIME_H" and also confirmed that #including <sys/time.h> fixes the warning.
I'm on OS X and building with CMake. I don't know enough about MINC's build system to know why HAVE_SYS_TIME_H is not true for me.
Help? :)
Thanks.
The error happens when mincresample is used with minc1 file on MacOSX, with libminc compiled as shared library.
While resampling a standard 3D volume with X,Y,Z dimension, mincresample tries to iterate over all possible dimensions and exclude their information from copying headers ( see https://github.com/BIC-MNI/minc-tools/blob/develop/progs/mincresample/mincresample.c#L1365 ).
By setting ncopts=0, it is supposed to prevent libminc from bailing, however on MacOSX the global variable ncopts inside mincresample points to a different location then from within libminc, which leads to failure when vector dimension is queried ( https://github.com/BIC-MNI/minc-tools/blob/develop/progs/mincresample/mincresample.c#L1393 ) and netcdf fails, causing mincresample to exit.
Looking with a debugger:
in the context of mincresample:
(lldb) p &ncopts
(int *) $2 = 0x00000001000cdf44
and in the context of libminc:
(lldb) p &ncopts
(int *) $4 = 0x00000001008c7be4
/opt/quarantine/ANTs/git/src/build/bin/PrintHeader brains/brain2_0.3_t1.mnc
HDF5-DIAG: Error detected in HDF5 (1.8.17) thread 0:
#000: /opt/quarantine/ANTs/git/src/build/ITKv4/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Dio.c line 173 in H5Dread(): can't read data
major: Dataset
minor: Read failed
#001: /opt/quarantine/ANTs/git/src/build/ITKv4/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Dio.c line 554 in H5D__read(): can't read data
major: Dataset
minor: Read failed
#002: /opt/quarantine/ANTs/git/src/build/ITKv4/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Dchunk.c line 1875 in H5D__chunk_read(): unable to read raw data chunk
major: Low-level I/O
minor: Read failed
#003: /opt/quarantine/ANTs/git/src/build/ITKv4/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Dchunk.c line 2905 in H5D__chunk_lock(): data pipeline read failed
major: Data filters
minor: Filter operation failed
#004: /opt/quarantine/ANTs/git/src/build/ITKv4/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5Z.c line 1349 in H5Z_pipeline(): required filter (name unavailable) is not registered
major: Data filters
minor: Read failed
#005: /opt/quarantine/ANTs/git/src/build/ITKv4/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5PL.c line 352 in H5PL_load(): search in paths failed
major: Plugin for dynamically loaded library
minor: Can't get value
#006: /opt/quarantine/ANTs/git/src/build/ITKv4/Modules/ThirdParty/HDF5/src/itkhdf5/src/H5PL.c line 456 in H5PL__find(): can't open directory
major: Plugin for dynamically loaded library
minor: Can't open directory or file
/opt/quarantine/ANTs/git/src/build/ITKv4/Modules/ThirdParty/MINC/src/libminc/libsrc2/hyper.c:620 (from MINC2): HDF5 function H5Dread failed
terminate called after throwing an instance of 'itk::ExceptionObject'
what(): /opt/quarantine/ANTs/git/src/build/ITKv4/Modules/IO/MINC/src/itkMINCImageIO.cxx:154:
itk::ERROR: MINCImageIO(0x1742100): Can not get real value hyperslab!!
I think this is due to an update to the version inside the headers?
mincdiff brain1_0.3_t1.mnc brain1_0.4_t1.mnc
1c1
< hdf5 brain1_0.3_t1 {
---
> hdf5 brain1_0.4_t1 {
3,5c3,5
< xspace = 555 ;
< yspace = 715 ;
< zspace = 566 ;
---
> xspace = 416 ;
> yspace = 536 ;
> zspace = 424 ;
7,10c7
< short image(zspace, yspace, xspace) ;
< image:version = "MINC Version 1.0" ;
< image:vartype = "group________" ;
< image:valid_range = 0., 65535. ;
---
> float image(zspace, yspace, xspace) ;
12c9,10
< image:complete = "true_" ;
---
> image:vartype = "group________" ;
> image:version = "MINC Version 2.0" ;
14c12
< image:signtype = "unsigned" ;
---
> image:valid_range = 0., 999.827087402344 ;
16,17d13
< image-min:version = "MINC Version 1.0" ;
< image-min:vartype = "var_attribute" ;
18a15,16
> image-min:vartype = "group________" ;
> image-min:version = "MINC Version 2.0" ;
20,21d17
< image-max:version = "MINC Version 1.0" ;
< image-max:vartype = "var_attribute" ;
22a19,20
> image-max:vartype = "group________" ;
> image-max:version = "MINC Version 2.0" ;
24c22
< xspace:version = "MINC Version 1.0" ;
---
> xspace:version = "MINC Version 2.0" ;
26c24
< xspace:vartype = "dimension____" ;
---
> xspace:vartype = "group________" ;
29d26
< xspace:alignment = "centre" ;
35,36c32,33
< xspace:length = 555 ;
< xspace:step = 0.3 ;
---
> xspace:length = 416 ;
> xspace:step = 0.4 ;
38c35
< yspace:version = "MINC Version 1.0" ;
---
> yspace:version = "MINC Version 2.0" ;
40c37
< yspace:vartype = "dimension____" ;
---
> yspace:vartype = "group________" ;
43d39
< yspace:alignment = "centre" ;
49,50c45,46
< yspace:length = 715 ;
< yspace:step = 0.3 ;
---
> yspace:length = 536 ;
> yspace:step = 0.4 ;
52c48
< zspace:version = "MINC Version 1.0" ;
---
> zspace:version = "MINC Version 2.0" ;
Passing the file through some older minc-toolkit version to cleanup the header fixes reading.
libminc-libminc-2-3-00/testdir/minc2-read-rgb.c:121]: (style) Variable 'start' is allocated memory that is never used.
libminc-libminc-2-3-00/testdir/minc2-read-rgb.c:122]: (style) Variable 'count' is allocated memory that is never used.
libminc-libminc-2-3-00/testdir/minc2-read-rgb.c:123]: (style) Variable 'howfar' is allocated memory that is never used.
I've set:
LIBMINC_MINC1_SUPPORT ON
NETCDF_INCLUDE_DIR /Users/builder/official_builds/netcdf-Universal/Debug/include
NETCDF_LIBRARY /Users/builder/official_builds/netcdf-Universal/Debug/lib/libnetcdf.a
then and CMake configure & generate work, but then make gives:
/Users/builder/external/libminc/libsrc/minc.h:171:10: fatal error: 'netcdf.h' file not found
#include <netcdf.h>
^
I'm quite sure my paths to NetCDF are good. I tried changing the #include to "" instead of <> but no help there.
Am I just missing a setting?
I'm trying to build libminc on debian stretch and it is failing. The FindHDF5 module does not look in the right places to find hdf5 installed through apt.
The serial include and lib directories for 64 bit install are:
/usr/include/hdf5/serial
/usr/lib/x86_64-linux-gnu/hdf5/serial
The cmake module FindHDF5 provided by kitware (https://github.com/Kitware/CMake/blob/master/Modules/FindHDF5.cmake) is able to find it by using the compiler wrappers h5cc/h5pcc, but seems to require SelectLibraryConfigurations.cmake and FindPackageHandleStandardArgs.cmake and their dependencies.
During my odyssey into fixing the on disk format for HDF5 1.10.x I discovered one other issue:
HDF5 1.10.0 and 1.10.1 do not have the H5F_LIBVER_V18 feature, so it is not possible via the current fix to handle them.
There are two other solutions
There's just one (well, two, but they are the same) -Wcast-qual warning left after my latest patches... this one I have no idea how to fix... thoughts?
nal/libminc/libsrc/ParseArgv.c:320:35: warning: cast from 'const char *' to 'int (*)()' drops const qualifier [-Wcast-qual]
handlerProc = (int (*)())infoPtr->src;
^
/libminc/libsrc/ParseArgv.c:332:35: warning: cast from 'const char *' to 'int (*)()' drops const qualifier [-Wcast-qual]
handlerProc = (int (*)())infoPtr->src;
^
Hello,
there is a bug in the Debian bug tracking system concerning building libminc on mipsel architecture. It is about two failed tests out of (49 tests are passing). When looking at the output it looks like a typical integer overflow:
Start 1: ezminc_rw_test
1/51 Test #1: ezminc_rw_test ................... Passed 0.23 sec
Start 2: minc_types-1
2/51 Test #2: minc_types-1 .....................***Failed 0.15 sec
364c364
< image[1] = 255
---
> image[1] = 0
373c373
< image[1] = 127
---
> image[1] = -128
382c382
< image[1] = 65535
---
> image[1] = 0
391c391
< image[1] = 32767
---
> image[1] = -32768
400c400
< image[1] = 2.14748e+09
---
> image[1] = 0
409c409
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
418c418
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
427c427
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
481c481
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
553c553
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
Start 3: minc_types-2
3/51 Test #3: minc_types-2 .....................***Failed 0.93 sec
364c364
< image[1] = 255
---
> image[1] = 0
373c373
< image[1] = 127
---
> image[1] = -128
382c382
< image[1] = 65535
---
> image[1] = 0
391c391
< image[1] = 32767
---
> image[1] = -32768
400c400
< image[1] = 2.14748e+09
---
> image[1] = 0
409c409
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
418c418
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
427c427
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
481c481
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
553c553
< image[1] = 2.14748e+09
---
> image[1] = -2.14748e+09
Start 4: icv_dim1-1
Here you can find the full build log (the issue can be found at the end):
https://buildd.debian.org/status/fetch.php?pkg=libminc&arch=mipsel&ver=2.3.00-2&stamp=1448468687
The according bug in the Debian BTS is
https://bugs.debian.org/806243
Any help to fix this would be welcome.
Kind regards
Andreas.
Available at https://scan.coverity.com/projects/1201
These 2 files have many warnings of the form:
/libminc/volume_io/Volumes/input_nifti.c:88:25: warning: cast from 'char *' to 'short *' increases required alignment from 1 to 2 [-Wcast-align]
tmp = (double) ((short *)data)[j];
^~~~~~~~~~~~~
Note that 'data' is defined as:
char data[CHUNK_SIZE];
Which means it may be aligned as char (1 byte).
This is not just pedantic, but a real bug that will expose itself on some architectures (I think ex: ARM) where unaligned access is not allowed. i.e. short* must be aligned in memory on the size of short (2 bytes).
One fix would be to use memcpy like:
short temp;
memcpy(&temp, (data + j*sizeof(short)), sizeof(short));
tmp = (double)temp;
Another might be to allocate 'data' with malloc() so that at least it's guaranteed to be aligned suitably for anything from char* to double*. This will fix the bug at runtime, but probably not get rid of the warning.
Other ideas?
I've found that using the ITK MINC tools (such as using ANTs built with MINC support) produce far larger files than using native MINC tools. Presumably this is because of compression, as if I use mincconvert to compress the files, they get very small.
Is there a way to convince the ITK tools to use MINC's internal compression?
The M4 files for tag release-1.5.00 are not available. When trying to build after checking this out the message says:
[melrobin@scorpion libminc]$ ./autogen.sh
The required m4 files were not found.
You need to check these out from their repository
using
cvs -d /software/source checkout -d m4 libraries/mni-acmacros
then the CVS command fails:
[melrobin@scorpion libminc]$ cvs -d /software/source checkout -d m4 libraries/mni-acmacros
cvs [checkout aborted]: /software/source/CVSROOT: No such file or directory
How can I check out the source to build this version?
Code for this version of libminc does not seem to do much with VIO_PREFIX_NAMES. Previous versions had typedefs to not break software using the older namespace. The line in volume_io/Include/volume_io.h:#define VIO_PREFIX_NAMES 1 /* Don't allow old-fashioned namespace pollution */
seems to want to address this, but there are no other points in the library that actually use this.
In particular the development version of Freesurfer uses General_transform, but libminc only has VIO_General_transform. There are a number of other datatypes that are broken by VIO_PREFIX_NAMES not working.
libminc2 API seem to be producing minc files without some descriptive fields , i.e :version and :varid is missing - is it important?
I have defined the paths properly for netcdf during my configuration of minc-toolkit:
CMakeCache.txt
NETCDF_INCLUDE_DIR:PATH=/scinet/gpc/Libraries/netcdf-4.2.1.1/serial_intel/include
NETCDF_LIBRARY:FILEPATH=/scinet/gpc/Libraries/netcdf-4.2.1.1/serial_intel/lib/libnetcdf.so
But when I get to libcminc during the build, the following issue arises:
[ 0%] Building C object libminc/CMakeFiles/minc2.dir/libsrc2/volume.c.o
In file included from /home/m/mchakrav/gdevenyi/projects/quarantine/minc-toolkit/git/src/minc-toolkit/libminc/libsrc2/volume.c(23):
/home/m/mchakrav/gdevenyi/projects/quarantine/minc-toolkit/git/src/minc-toolkit/libminc/libsrc/minc.h(171): catastrophic error: cannot open source file "netcdf.h"
#include <netcdf.h>
^
compilation aborted for /home/m/mchakrav/gdevenyi/projects/quarantine/minc-toolkit/git/src/minc-toolkit/libminc/libsrc2/volume.c (code 4)
make[2]: *** [libminc/CMakeFiles/minc2.dir/libsrc2/volume.c.o] Error 4
make[1]: *** [libminc/CMakeFiles/minc2.dir/all] Error 2
make: *** [all] Error 2
I believe this means the cmake doesn't seem to be inheriting the netcdf paths.
The build error fixed by 9d9c184 needs to be integrated into ITK, so perhaps a resync is in order.
That would also allow me to do a PR against ANTs to resolve ANTsX/ANTs#270
I reported this on the list ages ago, but...
/libminc/libsrc2/dimension.c:638:3: Value stored to 'record_flag' is never read
I think that 2nd parameter should be a pointer, to return by reference...
I see a lot of churn in the libminc code fixing various things, ITK pulls in libminc a77da29, perhaps when minc-toolkit-v2 drops, update ITK libminc dependency as well.
znzlib.c and nifti1_io.c emit quite a few warnings, but I hesitate to fix them because they are 3rd party. Is that code maintained by upstream developers? Do they take patches? Should we just use a #pragma to disable certain warnings in those files?
I'd like to do something, so that my dashboards can be green... :) BTW: I've set up some nightly libminc builds here:
UBSan finds only one test failure (nice!):
The test "minc2-hyper-test-2" has this loop:
59 for ( i = 0; i < CZ * CY * CX; i++ ) {
-> 60 buf[i] = ( unsigned short ) (i * 0.01);
61 }
Also note:
#define CZ 142
#define CY 245
#define CX 210
Thus the loop runs 7 305 900 times, which even after dividing by 100 gives numbers too big to fit in an unsigned 16 bit number and UBSan warns:
41: (/Users/builder/external/libminc-clang-dbg-x86_64-static/testdir/minc2-hyper-test-2+0x100004e36): runtime error: value 65536 is outside the range of representable values of type 'unsigned short'
Does the buffer contents matter much? If I divide by 1000 instead the test passed and the warning is gone.
m2util.c contains
#if (H5_VERS_MAJOR > 1) || (H5_VERS_MINOR > 6) || (H5_VERS_RELEASE > 2)
Is that trying to check for HDF5 > 1.6.2, because it's not... :)
input_nifti.h and input_mgh.h have many -Wdocumentation warnings because they have many empty \param commands.
Should they just be removed (I can do that) or can someone actually fill in docs?
(I'd like for my ITK dashboard to not have to suppress this warning flag...)
==> Downloading https://www.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8.15-patch1/sr
==> Downloading from https://support.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8.15-
Error: Failed to download resource "hdf5"
Download failed: https://www.hdfgroup.org/ftp/HDF5/releases/hdf5-1.8.15-patch1/src/hdf5-1.8.15-patch1.tar.bz2
The command "./brew/brew_install.sh" failed and exited with 1 during .
11 warnings about missing prototypes.
There are two solutions:
I'm not in a position to know really which is which...
/Users/sean/external/libminc/libsrc2/minc2_error.c:342:5: No previous prototype for function 'mi2log_set_verbosity'
/Users/sean/external/libminc/libsrc2/convert.c:269:1: No previous prototype for function 'miconvert_world_origin_to_start'
/Users/sean/external/libminc/libsrc2/convert.c:284:1: No previous prototype for function 'miconvert_spatial_frequency_origin_to_start'
/Users/sean/external/libminc/libsrc2/convert.c:448:1: No previous prototype for function 'miset_spatial_frequency_origin'
/Users/sean/external/libminc/volume_io/Volumes/input_volume.c:33:1: No previous prototype for function 'print_volume'
/Users/sean/external/libminc/volume_io/Volumes/output_mnc2.c:812:21: No previous prototype for function 'output_minc2_hyperslab'
/Users/sean/external/libminc/volume_io/Volumes/output_mnc2.c:1281:21: No previous prototype for function 'output_volume_to_minc2_file_position'
/Users/sean/external/libminc/volume_io/Volumes/input_mnc2.c:48:20: No previous prototype for function 'initialize_minc_input_from_minc2_id'
/Users/sean/external/libminc/volume_io/Volumes/input_mnc2.c:626:22: No previous prototype for function 'input_minc2_hyperslab'
/Users/sean/external/libminc/volume_io/Volumes/input_mnc2.c:967:19: No previous prototype for function 'advance_input_volume2'
/Users/sean/external/libminc/volume_io/Volumes/input_mnc2.c:1051:15: No previous prototype for function 'reset_input_volume2'
Most of our code works on large numbers of files and reads in a single (or 10) slice(s) of each of the files, performs some calculation, and moves on. The assumption being that the memory usage of those programs shouldn't really exceed number_files * 1 * length_y_dim * length_z_dim * 8 bytes. Often the memory used is orders of magnitude larger.
I've written a very simply c program that repeatedly reads the same 10x10x10 hyperslab from a fairly large file (482x956x630). The only malloc in the code is for that buffer (i.e. 8MB). However, when you run this program, its memory usage steadily increases. Does this indeed point to a memory leak related to miget_real_value_hyperslab(), or is there something wrong in this script?
#include <stdlib.h>
#include <minc2.h>
int main(int argc, char *argv[]) {
int result;
mihandle_t hvol;
int ndims;
midimhandle_t dimensions[5];
misize_t sizes[5];
misize_t count[5];
misize_t start[3];
double *buffer;
if( argc != 2) {
printf("Please supply a MINC input file\n");
return 0;
}
result = miopen_volume(argv[1], MI2_OPEN_READ, &hvol);
if (result != MI_NOERROR) {
fprintf(stderr, "\nError: opening the input file: %s\n", argv[1]);
return -1;
}
// get some information about the volume
result = miget_volume_dimension_count(hvol, MI_DIMCLASS_ANY, MI_DIMATTR_ALL, &ndims);
result = miget_volume_dimensions(hvol, MI_DIMCLASS_ANY, MI_DIMATTR_ALL, MI_DIMORDER_FILE, ndims, dimensions);
result = miget_dimension_sizes(dimensions, ndims, sizes);
fprintf(stdout, "Dim sizes : %llu, %llu, %llu\n", sizes[0], sizes[1], sizes[2]);
start[0] = start[1] = start[2] =0;
count[0] = count[1] = count[2] = 10;
buffer = (double *) malloc( 10 * 10 * 10 * sizeof(double));
for(int i=0; i < 10000000; i++){
result = miget_real_value_hyperslab(hvol, MI_TYPE_DOUBLE, start, count, buffer);
if (result != MI_NOERROR) {
fprintf(stderr, "\nError while getting hyperslab\n");
return -1;
}
if( i % 100 == 0) {
fprintf(stdout, ". ");
fflush(stdout);
}
}
fprintf(stdout, "\nDone.\n");
miclose_volume(hvol);
return result;
}
MINC2 reader/writer for volume IO is not producing exactly the same results as MINC1 based
The following tests FAILED:
25 - test_xfm_3 (SEGFAULT)
26 - copy_xfm (SEGFAULT)
27 - verify_xfm_1 (SEGFAULT)
28 - verify_xfm_float (SEGFAULT)
29 - verify_xfm_short (SEGFAULT)
30 - verify_xfm_byte (SEGFAULT)
31 - verify_xfm_2 (Failed)
(gdb) t a a bt full
Thread 1 (Thread 0x7ffff7fa5800 (LWP 32198)):
#0 0x00007ffff3cc1f59 in hdf_close () from /lib64/libnetcdf.so.7
No symbol table info available.
#1 0x00007ffff398eb51 in miclose (cdfid=16777216) at /home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/libsrc/netcdf_convenience.c:676
status = <optimized out>
#2 0x00007ffff39b0d4b in close_minc_input (file=0x5555557c95c0) at /home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/volume_io/Volumes/input_mnc.c:734
d = <optimized out>
#3 0x00007ffff39b4da1 in delete_volume_input (input_info=input_info@entry=0x7fffffffd8d0)
at /home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/volume_io/Volumes/input_volume.c:236
No locals.
#4 0x00007ffff39b4f42 in input_volume (filename=<optimized out>, n_dimensions=n_dimensions@entry=4, dim_names=dim_names@entry=0x0,
volume_nc_data_type=volume_nc_data_type@entry=0, volume_signed_flag=volume_signed_flag@entry=0, volume_voxel_min=volume_voxel_min@entry=0,
volume_voxel_max=volume_voxel_max@entry=0, create_volume_flag=create_volume_flag@entry=1, volume=0x7fffffffda78, options=0x7fffffffdbd0)
at /home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/volume_io/Volumes/input_volume.c:381
status = VIO_OK
amount_done = 1
input_info = {file_format = MNC_FORMAT, minc_file = 0x5555557c95c0, volume_file = 0x7fffffffe1f4, slice_index = -9848, sizes_in_file = {93824994346384,
140737488345496, 93824994346144, 0, 93824994346576}, axis_index_from_file = {0, 1, 2, 3, 4}, file_data_type = VIO_NO_DATA_TYPE, one_file_per_slice = 1433765456,
directory = 0x7fffffffda58 "pquUUU", slice_filenames = 0x7fffffffdd10, slice_byte_offsets = 0x7ffff36178ac <free+76>,
byte_slice_buffer = 0x42 <error: Cannot access memory at address 0x42>, short_slice_buffer = 0x7ffff39a5306 <unrecord_ptr_alloc_check+38>,
generic_slice_buffer = 0x5555557584a0, min_value = 0, max_value = 4.6355706427892377e-310, header_info = 0x7fffffffd9b8}
progress = {force_one_line = 0, first_msg_displayed = 0, one_line_flag = 1, n_steps = 1000, n_dots_so_far = 0, total_n_dots = 63, start_time = 0, previous_time = 0,
update_rate = 20, sum_xy = 0, sum_xx = 0, title = 0x5555557dfa50 "", last_check_time = 0, check_every = 1, next_check_step = 1, last_check_step = 0}
volume_min = 0
volume_max = 0
#5 0x00007ffff399f6ea in input_one_transform (transform=0x7fffffffda80,
filename=0x7fffffffe1f4 "/home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/testdir/vio_xfm_test/t3.xfm", file=0x555555757200)
at /home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/volume_io/MNI_formats/gen_xf_io.c:583
n_points = -9392
str = 0x555555757170 "version"
linear_transform = {m = {{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {-87, -45, 55, 1}}}
options = {promote_invalid_to_zero_flag = 1, convert_vector_to_scalar_flag = 0, convert_vector_to_colour_flag = 0, dimension_size_for_colour_data = 3,
max_dimension_size_for_colour_data = 4, rgba_indices = {0, 1, 2, 3}, user_real_range = {0, 0}}
status = <optimized out>
transform_type = <optimized out>
inverse_flag = 0
n_dimensions = 32767
displacements = 0x7fffffffda74
volume_filename = 0x5555557584a0 "/home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/testdir/vio_xfm_test//t3_grid_0.mnc"
directory = <optimized out>
---Type <return> to continue, or q <return> to quit---
tmp_filename = 0x5555557584a0 "/home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/testdir/vio_xfm_test//t3_grid_0.mnc"
inverse = {type = 134352, inverse_flag = 0, linear_transform = 0x7ffff3614283 <_int_free+2867>, inverse_linear_transform = 0xffff800000002441, n_points = -9281,
n_dimensions = 32767, points = 0x1, displacements = 0x3100000006, displacement_volume = 0x0, displacement_volume_file = 0x0, user_data = 0x6e0000005b,
size_user_data = 0, user_transform_function = 0x0, user_inverse_transform_function = 0x7c00000077, n_transforms = 1433760016, transforms = 0x7fffffffdc68}
i = <optimized out>
value = 55
points_1d = 0x7ffff7fab148
volume = 0x555555758710
j = <optimized out>
points = 0x7ffff3597d58
type_name = 0x555555757170 "version"
#6 input_transform (file=0x555555757200, filename=0x7fffffffe1f4 "/home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/testdir/vio_xfm_test/t3.xfm", transform=<optimized out>)
at /home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/volume_io/MNI_formats/gen_xf_io.c:660
n_transforms = 1
next = {type = LINEAR, inverse_flag = 0, linear_transform = 0x555555757030, inverse_linear_transform = 0x5555557570c0, n_points = -134576976, n_dimensions = 32767,
points = 0x7, displacements = 0x64, displacement_volume = 0xffff800000002501, displacement_volume_file = 0x7fffffffdaff "", user_data = 0x1,
size_user_data = 210453397511, user_transform_function = 0x0, user_inverse_transform_function = 0x0, n_transforms = 91, transforms = 0x0}
concated = {type = LINEAR, inverse_flag = 0, linear_transform = 0x7c00000077, inverse_linear_transform = 0x6e0000005b, n_points = 0, n_dimensions = 0, points = 0x0,
displacements = 0x7c00000077, displacement_volume = 0x1f7ffe6c0, displacement_volume_file = 0x7ffff3957b40 <main_arena> "", user_data = 0x64,
size_user_data = 93824994341120, user_transform_function = 0x7fff00000006, user_inverse_transform_function = 0x4e, n_transforms = -9280, transforms = 0x7ffff39e1009}
transform = <optimized out>
filename = 0x7fffffffe1f4 "/home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/testdir/vio_xfm_test/t3.xfm"
file = 0x555555757200
line = 0x555555757170 "version"
#7 0x00007ffff399f9af in input_transform_file (filename=0x7fffffffe1f4 "/home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/testdir/vio_xfm_test/t3.xfm",
transform=0x7fffffffdd10) at /home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/volume_io/MNI_formats/gen_xf_io.c:753
status = <optimized out>
file = 0x555555757200
#8 0x0000555555554a2f in main (ac=<optimized out>, av=0x7fffffffde88) at /home/brain/rpmbuild/BUILD/libminc-libminc-2-3-00/testdir/vio_xfm_test/copy-xfm.c:21
xfm = {type = LINEAR, inverse_flag = 0, linear_transform = 0x555555757030, inverse_linear_transform = 0x5555557570c0, n_points = -134576976, n_dimensions = 32767,
points = 0x7, displacements = 0x64, displacement_volume = 0xffff800000002501, displacement_volume_file = 0x7fffffffdaff "", user_data = 0x1,
size_user_data = 210453397511, user_transform_function = 0x0, user_inverse_transform_function = 0x0, n_transforms = 91, transforms = 0x0}
Specifically micreate uses a common cmode parameter that is used to pick the IO backend. However, this parameter also houses the back-end flags and is passed without changes to the selected backend.
In the case of MI2_CREATE_V1 this is a fatal mistake, since this results in calling nccreate, and in netcdf the value of MI2_CREATE_V1 0x2000 is actually interpreted as NC_MPIPOSIX, which in the case of netcdf-4.6 results in either failure to open the file (if parallel support was not compiled in) or other unknown consequences, if parallel support was compiled in.
See also https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=888925
Just these two left:
/libminc/libsrc/hdf_convenience.c:1850:20: Comparison of integers of different signs: 'long' and 'hsize_t' (aka 'unsigned long long')
/libminc/libsrc/hdf_convenience.c:1854:36: Comparison of integers of different signs: 'long' and 'hsize_t' (aka 'unsigned long long')
Is the left side expected to basically always be unsigned?
MINC2 reader/writer for volume IO is not producing exactly the same results as MINC1 based
INSTALL is still automake/autoconf centric.
It needs updating with simple instructions on how to build the package using cmake from a base .tar.gz. I am guessing this will be an out of tree build unlike the old in-tree autoconf/make build style.
test testdir/test_xfm 1000000 testdir/vio_xfm_test/t3.xfm 0.9 - fails
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.