silx-kit / hdf5plugin Goto Github PK

Set of compression filters for h5py

Home Page: http://www.silx.org/doc/hdf5plugin/latest/

License: Other

Python 0.52% Shell 1.09% C 49.74% Batchfile 0.01% CMake 1.21% Makefile 1.22% C++ 44.78% Starlark 0.03% Fortran 0.28% Cuda 0.15% HTML 0.07% Roff 0.07% Cython 0.15% SAS 0.01% Ada 0.21% Assembly 0.11% Pascal 0.18% C# 0.13% M4 0.02% DIGITAL Command Language 0.06%

hdf5plugin's People

Contributors

Stargazers

Watchers

hdf5plugin's Issues

Master Does Not Build on MacOSX

Current master commit (958c918) does not build on MacOSX)

MacOSX: 10.14.6
GCC / G++: clang 11.0.0
CMAKE: 3.14.0
Python: 3.7.4

Create new conda environment containing

python
pip (upgrade pip, setuptools, wheel to latest)

activate and pip install into env:

numpy
h5py

From hdf5plugin/ directory run $ python setup.py install

Exits with following error:

$ python setup.py install
running install
running bdist_egg
running egg_info
writing hdf5plugin.egg-info/PKG-INFO
writing dependency_links to hdf5plugin.egg-info/dependency_links.txt
writing requirements to hdf5plugin.egg-info/requires.txt
writing top-level names to hdf5plugin.egg-info/top_level.txt
reading manifest file 'hdf5plugin.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'setup.cfg'
writing manifest file 'hdf5plugin.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.9-x86_64/egg
running install_lib
running build_py
creating build/lib.macosx-10.9-x86_64-3.7
creating build/lib.macosx-10.9-x86_64-3.7/hdf5plugin
copying hdf5plugin/__init__.py -> build/lib.macosx-10.9-x86_64-3.7/hdf5plugin
copying hdf5plugin/test.py -> build/lib.macosx-10.9-x86_64-3.7/hdf5plugin
copying version.py -> build/lib.macosx-10.9-x86_64-3.7/hdf5plugin/_version.py
running build_ext
building 'hdf5plugin.plugins.libh5lz4' extension
creating build/temp.macosx-10.9-x86_64-3.7
creating build/temp.macosx-10.9-x86_64-3.7/src
creating build/temp.macosx-10.9-x86_64-3.7/src/LZ4
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/lz4-1.9.1
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/hdf5_dl.c -o build/temp.macosx-10.9-x86_64-3.7/src/hdf5_dl.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/LZ4/H5Zlz4.c -o build/temp.macosx-10.9-x86_64-3.7/src/LZ4/H5Zlz4.o
src/LZ4/H5Zlz4.c:24:9: warning: 'htonll' macro redefined [-Wmacro-redefined]
#define htonll(x) ( ( (uint64_t)(htonl( (uint32_t)((x << 32) >> 32)))<< 32) | htonl( ((uint32_t)(x >> 32)) ))
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/_endian.h:141:9: note: previous definition is here
#define htonll(x)       __DARWIN_OSSwapInt64(x)
        ^
src/LZ4/H5Zlz4.c:25:9: warning: 'ntohll' macro redefined [-Wmacro-redefined]
#define ntohll(x) htonll(x)
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/_endian.h:140:9: note: previous definition is here
#define ntohll(x)       __DARWIN_OSSwapInt64(x)
        ^
src/LZ4/H5Zlz4.c:205:5: warning: unused label 'done' [-Wunused-label]
    done:
    ^~~~~
src/LZ4/H5Zlz4.c:104:36: warning: comparison of integers of different signs: 'int' and 'uint32_t' (aka 'unsigned int') [-Wsign-compare]
                if(compressedBytes != compressedBlockSize)
                   ~~~~~~~~~~~~~~~ ^  ~~~~~~~~~~~~~~~~~~~
4 warnings generated.
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/lz4-1.9.1/lz4hc.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/lz4-1.9.1/lz4hc.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/lz4-1.9.1/lz4.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/lz4-1.9.1/lz4.o
creating build/lib.macosx-10.9-x86_64-3.7/hdf5plugin/plugins
gcc -bundle -undefined dynamic_lookup -L/Users/rick/miniconda3/envs/testhdf5/lib -arch x86_64 -L/Users/rick/miniconda3/envs/testhdf5/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.9-x86_64-3.7/src/hdf5_dl.o build/temp.macosx-10.9-x86_64-3.7/src/LZ4/H5Zlz4.o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/lz4-1.9.1/lz4hc.o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/lz4-1.9.1/lz4.o -o build/lib.macosx-10.9-x86_64-3.7/hdf5plugin/plugins/libh5lz4.dylib
building 'hdf5plugin.plugins.libh5bshuf' extension
creating build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle
creating build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src
creating build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/lz4
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/bitshuffle/src/ -Isrc/bitshuffle/lz4/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/hdf5_dl.c -o build/temp.macosx-10.9-x86_64-3.7/src/hdf5_dl.o -O3 -ffast-math -march=native -std=c99
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/bitshuffle/src/ -Isrc/bitshuffle/lz4/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/bitshuffle/src/bshuf_h5plugin.c -o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/bshuf_h5plugin.o -O3 -ffast-math -march=native -std=c99
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/bitshuffle/src/ -Isrc/bitshuffle/lz4/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/bitshuffle/src/bshuf_h5filter.c -o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/bshuf_h5filter.o -O3 -ffast-math -march=native -std=c99
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/bitshuffle/src/ -Isrc/bitshuffle/lz4/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/bitshuffle/src/bitshuffle.c -o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/bitshuffle.o -O3 -ffast-math -march=native -std=c99
src/bitshuffle/src/bitshuffle.c:61:14: warning: 'LZ4_compress' is deprecated: use LZ4_compress_default() instead [-Wdeprecated-declarations]
    nbytes = LZ4_compress((const char*) tmp_buf_bshuf, (char*) tmp_buf_lz4, size * elem_size);
             ^
/Users/rick/miniconda3/envs/testhdf5/include/lz4.h:453:12: note: 'LZ4_compress' has been explicitly marked deprecated here
LZ4LIB_API LZ4_DEPRECATED("use LZ4_compress_default() instead") int LZ4_compress               (const char* source, char* dest, int sourceSize);
           ^
/Users/rick/miniconda3/envs/testhdf5/include/lz4.h:437:52: note: expanded from macro 'LZ4_DEPRECATED'
#    define LZ4_DEPRECATED(message) __attribute__((deprecated(message)))
                                                   ^
1 warning generated.
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/bitshuffle/src/ -Isrc/bitshuffle/lz4/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/bitshuffle/src/bitshuffle_core.c -o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/bitshuffle_core.o -O3 -ffast-math -march=native -std=c99
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/bitshuffle/src/ -Isrc/bitshuffle/lz4/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/bitshuffle/src/iochain.c -o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/iochain.o -O3 -ffast-math -march=native -std=c99
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/bitshuffle/src/ -Isrc/bitshuffle/lz4/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/bitshuffle/lz4/lz4.c -o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/lz4/lz4.o -O3 -ffast-math -march=native -std=c99
gcc -bundle -undefined dynamic_lookup -L/Users/rick/miniconda3/envs/testhdf5/lib -arch x86_64 -L/Users/rick/miniconda3/envs/testhdf5/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.9-x86_64-3.7/src/hdf5_dl.o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/bshuf_h5plugin.o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/bshuf_h5filter.o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/bitshuffle.o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/bitshuffle_core.o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/src/iochain.o build/temp.macosx-10.9-x86_64-3.7/src/bitshuffle/lz4/lz4.o -o build/lib.macosx-10.9-x86_64-3.7/hdf5plugin/plugins/libh5bshuf.dylib
building 'hdf5plugin.plugins.libh5blosc' extension
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/blosc
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zstd-1.4.1
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zstd-1.4.1/compress
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zstd-1.4.1/dictBuilder
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zstd-1.4.1/decompress
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zstd-1.4.1/legacy
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zstd-1.4.1/common
creating build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zstd-1.4.1/deprecated
creating build/temp.macosx-10.9-x86_64-3.7/src/hdf5-blosc
creating build/temp.macosx-10.9-x86_64-3.7/src/hdf5-blosc/src
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/hdf5_dl.c -o build/temp.macosx-10.9-x86_64-3.7/src/hdf5_dl.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/blosc/blosc.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/blosc/blosc.o
src/c-blosc/blosc/blosc.c:446:14: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int' [-Wsign-compare]
  if (cbytes != compressed_length) {
      ~~~~~~ ^  ~~~~~~~~~~~~~~~~~
src/c-blosc/blosc/blosc.c:775:38: warning: comparison of integers of different signs: 'int32_t' (aka 'int') and 'unsigned long' [-Wsign-compare]
    if (src_offset < 0 || src_offset > compressedsize - sizeof(int32_t)) {
                          ~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/c-blosc/blosc/blosc.c:995:19: warning: comparison of integers of different signs: 'int32_t' (aka 'int') and 'unsigned long' [-Wsign-compare]
    if (blocksize > BLOSC_MAX_BLOCKSIZE) {
        ~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
src/c-blosc/blosc/blosc.c:1451:53: warning: comparison of integers of different signs: 'int32_t' (aka 'int') and 'size_t' (aka 'unsigned long') [-Wsign-compare]
  if (context->blocksize <= 0 || context->blocksize > destsize ||
                                 ~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~
src/c-blosc/blosc/blosc.c:1452:26: warning: comparison of integers of different signs: 'int32_t' (aka 'int') and 'unsigned long' [-Wsign-compare]
      context->blocksize > BLOSC_MAX_BLOCKSIZE || context->typesize <= 0 ||
      ~~~~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
src/c-blosc/blosc/blosc.c:1617:57: warning: comparison of integers of different signs: 'int32_t' (aka 'int') and 'unsigned long' [-Wsign-compare]
  if (blocksize <= 0 || blocksize > nbytes || blocksize > BLOSC_MAX_BLOCKSIZE ||
                                              ~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
6 warnings generated.
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/blosc/bitshuffle-generic.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/blosc/bitshuffle-generic.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/blosc/shuffle.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/blosc/shuffle.o
src/c-blosc/blosc/shuffle.c:298:4: warning: Hardware-acceleration detection not implemented for the target architecture. Only the generic shuffle/unshuffle routines will be available. [-W#warnings]
  #warning Hardware-acceleration detection not implemented for the target architecture. Only the generic shuffle/unshuffle routines will be available.
   ^
src/c-blosc/blosc/shuffle.c:308:22: warning: unused variable 'cpu_features' [-Wunused-variable]
  blosc_cpu_features cpu_features = blosc_get_cpu_features();
                     ^
In file included from src/c-blosc/blosc/shuffle.c:11:
src/c-blosc/blosc/shuffle-generic.h:32:13: warning: unused function 'shuffle_generic_inline' [-Wunused-function]
static void shuffle_generic_inline(const size_t type_size,
            ^
src/c-blosc/blosc/shuffle-generic.h:61:13: warning: unused function 'unshuffle_generic_inline' [-Wunused-function]
static void unshuffle_generic_inline(const size_t type_size,
            ^
4 warnings generated.
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/blosc/fastcopy.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/blosc/fastcopy.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/blosc/blosclz.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/blosc/blosclz.o
src/c-blosc/blosc/blosclz.c:101:17: warning: unused function 'get_run' [-Wunused-function]
static uint8_t *get_run(uint8_t *ip, const uint8_t *ip_bound, const uint8_t *ref) {
                ^
1 warning generated.
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/blosc/shuffle-generic.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/blosc/shuffle-generic.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/lz4-1.9.1/lz4hc.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/lz4-1.9.1/lz4hc.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/lz4-1.9.1/lz4.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/lz4-1.9.1/lz4.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/inflate.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/inflate.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/compress.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/compress.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/deflate.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/deflate.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/gzread.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/gzread.o
src/c-blosc/internal-complibs/zlib-1.2.8/gzread.c:30:15: warning: implicit declaration of function 'read' is invalid in C99 [-Wimplicit-function-declaration]
        ret = read(state->fd, buf + *have, len - *have);
              ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzread.c:30:15: warning: this function declaration is not a prototype [-Wstrict-prototypes]
src/c-blosc/internal-complibs/zlib-1.2.8/gzread.c:591:11: warning: implicit declaration of function 'close' is invalid in C99 [-Wimplicit-function-declaration]
    ret = close(state->fd);
          ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzread.c:591:11: warning: this function declaration is not a prototype [-Wstrict-prototypes]
4 warnings generated.
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/crc32.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/crc32.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/infback.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/infback.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/zutil.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/zutil.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.o
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:256:24: warning: implicit declaration of function 'lseek' is invalid in C99 [-Wimplicit-function-declaration]
        state->start = LSEEK(state->fd, 0, SEEK_CUR);
                       ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:256:24: warning: this function declaration is not a prototype [-Wstrict-prototypes]
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:355:9: warning: implicit declaration of function 'lseek' is invalid in C99 [-Wimplicit-function-declaration]
    if (LSEEK(state->fd, state->start, SEEK_SET) == -1)
        ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:396:15: warning: implicit declaration of function 'lseek' is invalid in C99 [-Wimplicit-function-declaration]
        ret = LSEEK(state->fd, offset - state->x.have, SEEK_CUR);
              ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:492:14: warning: implicit declaration of function 'lseek' is invalid in C99 [-Wimplicit-function-declaration]
    offset = LSEEK(state->fd, 0, SEEK_CUR);
             ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzlib.c:14:17: note: expanded from macro 'LSEEK'
#  define LSEEK lseek
                ^
5 warnings generated.
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/inftrees.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/inftrees.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/uncompr.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/uncompr.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/gzwrite.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/gzwrite.o
src/c-blosc/internal-complibs/zlib-1.2.8/gzwrite.c:84:15: warning: implicit declaration of function 'write' is invalid in C99 [-Wimplicit-function-declaration]
        got = write(state->fd, strm->next_in, strm->avail_in);
              ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzwrite.c:84:15: warning: this function declaration is not a prototype [-Wstrict-prototypes]
src/c-blosc/internal-complibs/zlib-1.2.8/gzwrite.c:101:33: warning: implicit declaration of function 'write' is invalid in C99 [-Wimplicit-function-declaration]
            if (have && ((got = write(state->fd, state->x.next, have)) < 0 ||
                                ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzwrite.c:573:9: warning: implicit declaration of function 'close' is invalid in C99 [-Wimplicit-function-declaration]
    if (close(state->fd) == -1)
        ^
src/c-blosc/internal-complibs/zlib-1.2.8/gzwrite.c:573:9: warning: this function declaration is not a prototype [-Wstrict-prototypes]
5 warnings generated.
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/trees.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/trees.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/gzclose.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/gzclose.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/inffast.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/inffast.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zlib-1.2.8/adler32.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zlib-1.2.8/adler32.o
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -I/Users/rick/miniconda3/envs/testhdf5/include -arch x86_64 -DH5_USE_18_API -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Isrc/hdf5/include -Isrc/hdf5/include/darwin -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -I/Users/rick/miniconda3/envs/testhdf5/include/python3.7m -c src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c -o build/temp.macosx-10.9-x86_64-3.7/src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.o
In file included from src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:28:
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstd_compress_internal.h:118:5: error: unknown type name 'ZSTD_literalCompressionMode_e'
    ZSTD_literalCompressionMode_e literalCompressionMode;
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstd_compress_internal.h:201:5: error: unknown type name 'ZSTD_dictAttachPref_e'
    ZSTD_dictAttachPref_e attachDictPref;
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstd_compress_internal.h:202:5: error: unknown type name 'ZSTD_literalCompressionMode_e'
    ZSTD_literalCompressionMode_e literalCompressionMode;
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:676:45: warning: implicit declaration of function 'ZSTD_CCtxParams_setParameter' is invalid in C99 [-Wimplicit-function-declaration]
        {   size_t const forceWindowError = ZSTD_CCtxParams_setParameter(&jobParams, ZSTD_c_forceMaxWindow, !job->firstJob);
                                            ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:676:45: warning: this function declaration is not a prototype [-Wstrict-prototypes]
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:676:86: error: use of undeclared identifier 'ZSTD_c_forceMaxWindow'; did you mean 'ZSTD_p_forceMaxWindow'?
        {   size_t const forceWindowError = ZSTD_CCtxParams_setParameter(&jobParams, ZSTD_c_forceMaxWindow, !job->firstJob);
                                                                                     ^~~~~~~~~~~~~~~~~~~~~
                                                                                     ZSTD_p_forceMaxWindow
/Users/rick/miniconda3/envs/testhdf5/include/zstd.h:1110:5: note: 'ZSTD_p_forceMaxWindow' declared here
    ZSTD_p_forceMaxWindow=1100, /* Force back-reference distances to remain < windowSize,
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:868:12: warning: implicit declaration of function 'ZSTD_CCtxParams_setParameter' is invalid in C99 [-Wimplicit-function-declaration]
    return ZSTD_CCtxParams_setParameter(params, ZSTD_c_nbWorkers, (int)nbWorkers);
           ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:868:49: error: use of undeclared identifier 'ZSTD_c_nbWorkers'; did you mean 'ZSTD_p_nbWorkers'?
    return ZSTD_CCtxParams_setParameter(params, ZSTD_c_nbWorkers, (int)nbWorkers);
                                                ^~~~~~~~~~~~~~~~
                                                ZSTD_p_nbWorkers
/Users/rick/miniconda3/envs/testhdf5/include/zstd.h:1091:5: note: 'ZSTD_p_nbWorkers' declared here
    ZSTD_p_nbWorkers=400,    /* Select how many threads will be spawned to compress in parallel.
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:997:16: warning: implicit declaration of function 'ZSTD_CCtxParams_setParameter' is invalid in C99 [-Wimplicit-function-declaration]
        return ZSTD_CCtxParams_setParameter(params, ZSTD_c_jobSize, value);
               ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:997:53: error: use of undeclared identifier 'ZSTD_c_jobSize'; did you mean 'ZSTD_p_jobSize'?
        return ZSTD_CCtxParams_setParameter(params, ZSTD_c_jobSize, value);
                                                    ^~~~~~~~~~~~~~
                                                    ZSTD_p_jobSize
/Users/rick/miniconda3/envs/testhdf5/include/zstd.h:1098:5: note: 'ZSTD_p_jobSize' declared here
    ZSTD_p_jobSize,          /* Size of a compression job. This value is enforced only in non-blocking mode.
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1000:53: error: use of undeclared identifier 'ZSTD_c_overlapLog'; did you mean 'ZSTDMT_p_overlapLog'?
        return ZSTD_CCtxParams_setParameter(params, ZSTD_c_overlapLog, value);
                                                    ^~~~~~~~~~~~~~~~~
                                                    ZSTDMT_p_overlapLog
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.h:113:5: note: 'ZSTDMT_p_overlapLog' declared here
    ZSTDMT_p_overlapLog,  /* Each job may reload a part of previous job to enhance compression ratio; 0 == no overlap, 6(default) == use 1/8th of window, >=9 == use full window. This is a "sticky" parameter : its value will be re-...
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1003:53: error: use of undeclared identifier 'ZSTD_c_rsyncable'; did you mean 'ZSTDMT_p_rsyncable'?
        return ZSTD_CCtxParams_setParameter(params, ZSTD_c_rsyncable, value);
                                                    ^~~~~~~~~~~~~~~~
                                                    ZSTDMT_p_rsyncable
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.h:114:5: note: 'ZSTDMT_p_rsyncable' declared here
    ZSTDMT_p_rsyncable    /* Enables rsyncable mode. */
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1019:16: warning: implicit declaration of function 'ZSTD_CCtxParams_getParameter' is invalid in C99 [-Wimplicit-function-declaration]
        return ZSTD_CCtxParams_getParameter(&mtctx->params, ZSTD_c_jobSize, value);
               ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1019:16: warning: this function declaration is not a prototype [-Wstrict-prototypes]
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1019:61: error: use of undeclared identifier 'ZSTD_c_jobSize'; did you mean 'ZSTD_p_jobSize'?
        return ZSTD_CCtxParams_getParameter(&mtctx->params, ZSTD_c_jobSize, value);
                                                            ^~~~~~~~~~~~~~
                                                            ZSTD_p_jobSize
/Users/rick/miniconda3/envs/testhdf5/include/zstd.h:1098:5: note: 'ZSTD_p_jobSize' declared here
    ZSTD_p_jobSize,          /* Size of a compression job. This value is enforced only in non-blocking mode.
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1021:61: error: use of undeclared identifier 'ZSTD_c_overlapLog'; did you mean 'ZSTDMT_p_overlapLog'?
        return ZSTD_CCtxParams_getParameter(&mtctx->params, ZSTD_c_overlapLog, value);
                                                            ^~~~~~~~~~~~~~~~~
                                                            ZSTDMT_p_overlapLog
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.h:113:5: note: 'ZSTDMT_p_overlapLog' declared here
    ZSTDMT_p_overlapLog,  /* Each job may reload a part of previous job to enhance compression ratio; 0 == no overlap, 6(default) == use 1/8th of window, >=9 == use full window. This is a "sticky" parameter : its value will be re-...
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1023:61: error: use of undeclared identifier 'ZSTD_c_rsyncable'; did you mean 'ZSTDMT_p_rsyncable'?
        return ZSTD_CCtxParams_getParameter(&mtctx->params, ZSTD_c_rsyncable, value);
                                                            ^~~~~~~~~~~~~~~~
                                                            ZSTDMT_p_rsyncable
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.h:114:5: note: 'ZSTDMT_p_rsyncable' declared here
    ZSTDMT_p_rsyncable    /* Enables rsyncable mode. */
    ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1172:14: error: use of undeclared identifier 'ZSTD_btultra2'; did you mean 'ZSTD_btultra'?
        case ZSTD_btultra2:
             ^~~~~~~~~~~~~
             ZSTD_btultra
/Users/rick/miniconda3/envs/testhdf5/include/zstd.h:450:42: note: 'ZSTD_btultra' declared here
               ZSTD_btlazy2, ZSTD_btopt, ZSTD_btultra } ZSTD_strategy;   /* from faster to stronger */
                                         ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1174:14: error: duplicate case value 'ZSTD_btultra'
        case ZSTD_btultra:
             ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1172:14: note: previous case defined here
        case ZSTD_btultra2:
             ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1993:16: warning: implicit declaration of function 'ZSTD_compressStream2' is invalid in C99 [-Wimplicit-function-declaration]
        return ZSTD_compressStream2(mtctx->cctxPool->cctx[0], output, input, endOp);
               ^
src/c-blosc/internal-complibs/zstd-1.4.1/compress/zstdmt_compress.c:1993:16: warning: this function declaration is not a prototype [-Wstrict-prototypes]
8 warnings and 13 errors generated.
error: command 'gcc' failed with exit status 1

Has anyone else run into this? Any solutions?

Adding MAFISC filter

Once JPEG is added (#85), adding BZIP2 and MAFISC would make hdf5plugin embedding all "commonly used Registered Filter Plugins [..] packaged into the HDF5 Plugin software" (see HDF5 Filter Plugins: BZIP2, JPEG, LZF, BLOSC, MAFISC, LZ4, Bitshuffle, and ZFP) except LZF which is already provided by h5py.

bitshuffle openmp support

As it is, the bitshuffle HDF5 filter is never built with openmp.

Timing for 2.0 release?

Hi team, just wondering what the timing for the v2.0 release will be? I saw the pre-release a few days, ago, and got very excited about being able to include it in the project I maintain in our upcoming 0.4 release (which I'd like to get out in the next two weeks or so).

If there's anything you could use some help on, I'd be happy to contribute a PR or two to help speed along the process. Just let me know!

Best,
Rick

Zpf does not work for uint8 arrays.

Hi.

Zpf gives the error : ValueError: Unable to create dataset (invalid datatype size) when running the following code:

import numpy
import h5py
import hdf5plugin

# Compression
f = h5py.File('test.h5', 'w')
array = numpy.uint8(numpy.random.randint(255, size=(100, 100)))
f.create_dataset('data', data=array, **hdf5plugin.Zfp())
f.close()

However, changing it to numpy.uint32 works. How can this be?
In your setup file you specify that it is build with uint8. So if i understand it correctly, it should work for uint8 but for uint32.

hdf5plugin/setup.py

Line 524 in 0cb2154

'cflags': ['-DBIT_STREAM_WORD_TYPE=uint8'],

Best regards

Add license conditions of used filters.

Some of the used filters ask the license/copyright to be distributed with the binary files.

Can't read file compressed with FCIDECOMP

This is a follow up of h5py/h5py#1537 somehow.

I just installed @vasole 's wheel: http://ftp.esrf.fr/pub/bliss/hdf5plugin-2.1.3a0-py2.py3-none-linux_x86_64.whl with pip, but still get an error when reading a sample compressed file:

❯ python
Python 3.8.2 | packaged by conda-forge | (default, Mar 23 2020, 18:16:37) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import h5py
>>> h5f = h5py.File('/home/a001673/usr/contrib/FCIDECOMP_V1.0.1/Software/FCIDECOMP_SOURCES/fcidecomp-test/data/sample.nc', mode='r')
>>> h5f["effective_radiance"]
<HDF5 dataset "effective_radiance": shape (60, 30), type "<i2">
>>> h5f["effective_radiance"][:]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/a001673/miniconda3/envs/py3/lib/python3.8/site-packages/h5py/_hl/dataset.py", line 573, in __getitem__
    self.id.read(mspace, fspace, arr, mtype, dxpl=self._dxpl)
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5d.pyx", line 182, in h5py.h5d.DatasetID.read
  File "h5py/_proxy.pyx", line 130, in h5py._proxy.dset_rw
  File "h5py/_proxy.pyx", line 84, in h5py._proxy.H5PY_H5Dread
OSError: Can't read data (can't open directory: /home/a001673/miniconda3/envs/py3/lib/hdf5/plugin)

As you can see, this is running in conda environment, with python 3.8.

Any more details I can provide ?

Arm support

Is there any chance for ARM/ARM64 support?

Issue for building on Power9

powerpc64le-linux-gnu-gcc: error: unrecognized command line option ‘-msse2’; did you mean ‘-misel’?
WARNING:__main__:SSE2 disabled: not available
creating /tmp/tmpn_n0d0xt/tmp
creating /tmp/tmpn_n0d0xt/tmp/tmpn_n0d0xt
powerpc64le-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -fexceptions -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -c /tmp/tmpn_n0d0xt/source.c -o /tmp/tmpn_n0d0xt/tmp/tmpn_n0d0xt/source.o -mavx2
powerpc64le-linux-gnu-gcc: error: unrecognized command line option ‘-mavx2’
WARNING:__main__:AVX2 disabled: not available
creating /tmp/tmp0ihr6rm_/tmp
creating /tmp/tmp0ihr6rm_/tmp/tmp0ihr6rm_
powerpc64le-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -fexceptions -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -c /tmp/tmp0ihr6rm_/source.c -o /tmp/tmp0ihr6rm_/tmp/tmp0ihr6rm_/source.o -openmp
Traceback (most recent call last):
  File "setup.py", line 519, in <module>
    libraries=libraries,
  File "/home/kieffer/.venv/py37/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.7/distutils/dist.py", line 984, in run_command
    cmd_obj.ensure_finalized()
  File "/usr/lib/python3.7/distutils/cmd.py", line 107, in ensure_finalized
    self.finalize_options()
  File "setup.py", line 184, in finalize_options
    is_cpu_sse2, is_cpu_avx2 = get_cpu_sse2_avx2()
  File "setup.py", line 83, in get_cpu_sse2_avx2
    cpu_flags = cpuinfo.get_cpu_info()['flags']
KeyError: 'flags'

Add test suite to generated wheel.

Add Zstandard HDF5 plugin

Would it be possible to add Zstandard (https://facebook.github.io/zstd/) in the form of an HDF5 plugin into your repository? An already possible integration can be found here: https://github.com/aparamon/HDF5Plugin-Zstandard. Thanks and keep up the good work! :)

build error on MacOS/ARM 3.8

I'm trying to build hdf5plugin 2.3.2 on MacOS on ARM64 for conda-forge (conda-forge/hdf5plugin-feedstock#20), and I see the following build error:

arm64-apple-darwin20.0.0-clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O3 -Wall -Wstrict-prototypes -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O3 -pipe -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/hdf5plugin_1622044221898/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol=/usr/local/src/conda-prefix -flto -Wl,-export_dynamic -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/runner/miniforge3/conda-bld/hdf5plugin_1622044221898/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/hdf5plugin_1622044221898/work=/usr/local/src/conda/hdf5plugin-2.3.2 -fdebug-prefix-map=/Users/runner/miniforge3/conda-bld/hdf5plugin_1622044221898/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol=/usr/local/src/conda-prefix -D_FORTIFY_SOURCE=2 -isystem /Users/runner/miniforge3/conda-bld/hdf5plugin_1622044221898/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include -mmacosx-version-min=11.0 -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -DH5_USE_18_API -DHAVE_SNAPPY=1 -DSHUFFLE_SSE2_ENABLED=1 -I/Users/runner/miniforge3/conda-bld/hdf5plugin_1622044221898/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.1 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.1 -Isrc/c-blosc/internal-complibs/zstd-1.4.1/common -Isrc/hdf5-blosc/src/ -Isrc/c-blosc/internal-complibs/snappy-1.1.1 -I/Users/runner/miniforge3/conda-bld/hdf5plugin_1622044221898/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehol/include/python3.8 -c src/c-blosc/blosc/shuffle.c -o build/temp.macosx-11.0-arm64-3.8/src/c-blosc/blosc/shuffle.o -msse2
    clang-11: warning: -Wl,-export_dynamic: 'linker' input unused [-Wunused-command-line-argument]
    clang-11: warning: argument unused during compilation: '-msse2' [-Wunused-command-line-argument]
    src/c-blosc/blosc/shuffle.c:163:5: error: invalid output constraint '=b' in asm
        "=b" (cpuInfo[1]),
        ^
    src/c-blosc/blosc/shuffle.c:188:5: error: invalid output constraint '=a' in asm
        "=a" (eax),
        ^
    2 errors generated.
    error: command 'arm64-apple-darwin20.0.0-clang' failed with exit status 1

Is MacOS/ARM supported by hdf5plugin?

reference the sources of the plugins in documentatiion & readme

For info ..

Change helper API to prepare for better integration with h5py 3.0

Initially proposed here h5py/h5py#1315 (comment) by @takluyver:

f.create_dataset('blosc_byte_shuffle_blosclz', data=numpy.arange(100),
    compression=hdf5plugin.Blosc(level=9, shuffle='byte'))

# Instead of
f.create_dataset('blosc_byte_shuffle_blosclz', data=numpy.arange(100),
    **hdf5plugin.blosc_options(level=9, shuffle='byte', compression='blosclz'))

Basically turn *_options helpers to classes which can be provided as the compression argument of create_dataset with h5py 3.x... and still can be used as **kwargs for h5py v2.x

Update Wheelhouse

Either we add the new versions to the wheelhouse either we get rid of the version available there

http://www.silx.org/pub/wheelhouse/

Add compression helper functions

Create 2 helper functions (bitshuffle_options(nelem=0, lz4=False) and lz4_options(nbytes=0) for the options to provide to bitshuffle and lz4 filters as it already done for blosc with blosc_options.
This should replaces the variable BSHUF_LZ4_OPTS.

Documentation and tests needs to be updated accordingly.

add test reading blosc data

lz4 and bitshuffle are tested with a file written with a different software, but blosc is not.
This is missing.

Cut a 2.1.0 release

Update version
Update changelog

[CI] We are testing MacOS exclusively under Python 2

Only python3 testing is actually needed

Prepend or append to existing HDF5_PLUGIN_PATH

What should take priority, the eventual user supplied libraries or those supplied by this module?

Adding / importing JPEG filter

Hi.

I've been looking at this lossy jpeg filter : https://github.com/CARS-UChicago/jpegHDF5 which i'd very much like to try.
Is it possible to add it / import it with the hdf5plugin module?

Best regards
Peter

Have some CI set-up for linux and macos

With change from travis-ci.org to travis-ci.com, there is currently no longer CI running for this project.
This needs to be fixed one way or another.

Update c-blosc to version 1.20.1

Needed to build the plugin under MacOS and 3.8

Make manylinux2014 wheels

That would allow to enable snappy (C++11) for manylinux.
No need for a new release, the wheel can be added to pypi.

For info, see:

Update CHANGELOG and MANIFEST

Those files are not up-to-date.

FCIDECOMP Update version of the original sources

The test file of the 1.0.1 version uses wrong filter ID.

Add continuous integration tests.

No automatic testing built in yet.

blosc filter sse2/avx2 support

As it is, sse2/avx is not taken into account while compiling the HDF5 blosc filter.

Fail to build manylinux1 wheels

There is an issue building manylinux1 wheels with current master.
Compilation of cblosc fails:

gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DHAVE_LZ4=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -DH5_USE_18_API -DH5_USE_18_API -DSHUFFLE_SSE2_ENABLED=1 -Isrc/hdf5/include -Isrc/hdf5/include/linux -Isrc/c-blosc/ -Isrc/c-blosc/blosc -Isrc/c-blosc/internal-complibs/lz4-1.9.2 -Isrc/c-blosc/internal-complibs/zlib-1.2.8 -Isrc/c-blosc/internal-complibs/zstd-1.4.5 -Isrc/c-blosc/internal-complibs/zstd-1.4.5/common -Isrc/hdf5-blosc/src/ -I/opt/python/cp37-cp37m/include/python3.7m -c src/c-blosc/blosc/blosclz.c -o build/temp.linux-x86_64-3.7/src/c-blosc/blosc/blosclz.o -msse2
src/c-blosc/blosc/blosclz.c: In function ‘get_csize’:
src/c-blosc/blosc/blosclz.c:409:3: error: ‘for’ loop initial declarations are only allowed in C99 mode
   for (unsigned i = 0; i < (1U << HASH_LOG); i++) {
   ^
src/c-blosc/blosc/blosclz.c:409:3: note: use option -std=c99 or -std=gnu99 to compile your code
src/c-blosc/blosc/blosclz.c: In function ‘blosclz_compress’:
src/c-blosc/blosc/blosclz.c:534:3: error: ‘for’ loop initial declarations are only allowed in C99 mode
   for (unsigned i = 0; i < (1U << hashlog); i++) {
   ^
src/c-blosc/blosc/blosclz.c:643:47: warning: signed and unsigned type in conditional expression [-Wsign-compare]
     unsigned minlen = (clevel == 9) ? ipshift : minlen_[clevel];
                                               ^
error: command 'gcc' failed with exit status 1

Full log: https://gitlab.esrf.fr/silx/bob/hdf5plugin/-/jobs/174560

ValueError: Compression filter "<class 'hdf5plugin.Blosc'>" is unavailable

h5py/h5py#611 suggested this package might be a good way to get the Blosc filter to work, but:

$ python3.6 -m venv blosc_example
$ source blosc_example/bin/activate
(blosc_example) $ pip install hdf5plugin
...
Collecting hdf5plugin
  Downloading .../hdf5plugin-2.0.0-py2.py3-none-manylinux1_x86_64.whl (3.8MB)
    100% |████████████████████████████████| 3.8MB 37.5MB/s
...
Installing collected packages: numpy, six, h5py, hdf5plugin
Successfully installed h5py-2.10.0 hdf5plugin-2.0.0 numpy-1.17.3 six-1.13.0
(blosc_example) .$ python
Python 3.6.7 (default, Dec  5 2018, 15:01:38)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import h5py, numpy as np, hdf5plugin
>>> f = h5py.File('test.h5', 'w')
>>> f.create_dataset('data', data=np.arange(100), compression=hdf5plugin.Blosc)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/blosc_example/lib64/python3.6/site-packages/h5py/_hl/group.py", line 136, in create_dataset
    dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
  File "/tmp/blosc_example/lib64/python3.6/site-packages/h5py/_hl/dataset.py", line 140, in make_new_dset
    maxshape, scaleoffset, external)
  File "/tmp/blosc_example/lib64/python3.6/site-packages/h5py/_hl/filters.py", line 138, in fill_dcpl
    raise ValueError('Compression filter "%s" is unavailable' % compression)
ValueError: Compression filter "<class 'hdf5plugin.Blosc'>" is unavailabl

This is IUS Python 3.6 on RHEL7.7. What might be going wrong?

Make a 2.x pre-release

With recent changes and "bob the builder" ready to build the wheels: https://silx.gitlab-pages.esrf.fr/bob/hdf5plugin/ we can make a pre-release of 2.0.0.

Add configuration of zfp filter

See https://h5z-zfp.readthedocs.io/en/latest/interfaces.html

Building with C++11 fails on macOS

The -std=c++11 is not accepted for C files under macOS.

wheel content is stored in platlib subfolder

Investigate why hdf5plugin is stored in hdf5plugin-2.0.0.data/platlib/.
Check whether this is normal or not.
This is due to setting root_is_pure in the subclass of bdist_wheel that tweaks the wheel generation to make one wheel working for the different versions of python.

Initially raised here: https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!topic/h5py/EMLVilY8oHk

MacOS wheels do not work on Yosemite or El Capitan

The Python 3.6 wheels do not work on Yosemite or El Capitan machines.

Are you sure we are setting the MACOSX_DEPLOYMENT_TARGET to something acceptable?

Building the wheels on those machines works.

add the bitshuffle implementation used for new eiger detectors

Update changelog

Rename generated plugins to remove python specific naming

Example libh5blosc.cpython-37m-x86_64-linux-gnu.so should become libh5blosc.so

Request: Please add to Anaconda

I noticed that silx is in conda-forge. Please:
(1) Add hdf5plugin to either default or conda-forge.
(2) Document this in README.md.

Thank you.

h5py support for additional filters

Hi hdf5plugin developers

I came across this project via h5py/h5py#923, I wonder if you guys would be interested contributing support in h5py for detecting the installation of hdf5 filters via (something like) entry points, which would mean that users wouldn't have to worry about import order?

Support for zfp

Love hdf5 plugin. Use it for blossc. What would be great it so have lossy compression available. Is there a possibility of supporting zfp.

https://github.com/LLNL/H5Z-ZFP

Thanks!

Build options at runtime ?

When I import hdf5plugin is there a way to check if it was compiled for the current cpu and whether openmp is enabled?
Related, is there a plan to have these options selected at runtime ?
Thanks!

"Cannot initialize filter fcidecomp: File not found"

This message started appearing in the last several days.

Envirtonment
O/S: Ubuntu 20.10, up-to-date
hdf5plugin version: 2.3.2
h5py version: 3.2.1
Python 3.8.6
All of my packages are managed under pip 20.1.1.

I just recently observed the subject message which seems to originate from the hdf5plugin.

import h5py
import hdf5plugin
print('Right after importing hdf5plugin')

gets this stdout:

Cannot initialize filter fcidecomp: File not found
Right after importing hdf5plugin

Switch the order of the 2 imports, I see the same thing.

blosc filter add snappy support

snappy is currently not built in blosc.
It would be possible to build it optionally (through environment variable or build option) when C++11 is available.

Fill copyright file

Following PR #87, for the packaging to be fully correct, the copyright file needs to be filled to reflect all licenses and copyright in the repository.

macOS build segmentation fault

I cannot compile ZFP on macOS. Config:

macOS Sierra 10.12.6
Apple LLVM version 8.0.0 (clang-800.0.42.1)
XCode 8.2.1

This patch fixes the issue:

setup.py
@@ -521,7 +521,7 @@ zfp_include_dirs = [os.path.join(zfp_dir, 'include')]
 zfp_lib = ('zfp', {
     'sources': zfp_sources,
     'include_dirs': zfp_include_dirs,
-    'cflags': ['-DBIT_STREAM_WORD_TYPE=uint8'],
+    'cflags': ['-O1', '-DBIT_STREAM_WORD_TYPE=uint8'],
     })

It's build by default with -O3.

Error:

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g -Isrc/zfp/include -c src/zfp/src/encode2f.c -o build/temp.macosx-10.9-x86_64-3.8/src/zfp/src/encode2f.o -DBIT_STREAM_WORD_TYPE=uint8
clang: error: unable to execute command: Segmentation fault: 11
clang: error: clang frontend command failed due to signal (use -v to see invocation)
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
clang: note: diagnostic msg: PLEASE submit a bug report to http://developer.apple.com/bugreporter/ and include the crash backtrace, preprocessed source, and associated run script.
clang: note: diagnostic msg: 
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: encode2f-b40a40.c
clang: note: diagnostic msg: encode2f-b40a40.sh
clang: note: diagnostic msg: 

********************
error: command 'gcc' failed with exit status 254

Looks to be a llvm bug.

Crash with h5py3.0rc1 on macos

Tested on macos with python3.7, h5py and hdf5plugin installed from wheels.

Some filters crash with h5py 3.0rc1:

import h5py, hdf5plugin, numpy 
f = h5py.File("test.h5", "a")

f.create_dataset('data2', data=numpy.arange(643*922*3, dtype='uint8'), **hdf5plugin.Blosc())                                                                            
Python(1757,0x7fffafcbe3c0) malloc: *** error for object 0x7f8404b04c30: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
...
f.create_dataset('data2', data=numpy.arange(643*922*3, dtype='uint8'), **hdf5plugin.Bitshuffle())                                                                       
Python(1742,0x7fffafcbe3c0) malloc: *** error for object 0x7fbff9424430: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6
...
f.create_dataset('data2', data=numpy.arange(643*922*3, dtype='float32'), **hdf5plugin.Zfp())                                                                            
Python(1772,0x7fffafcbe3c0) malloc: *** error for object 0x7f8befc88e30: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort trap: 6

LZ4 and FciDecomp works for fine under same conditions.

I cannot reproduce the same with a python3.8 virtualenv on ubuntu20.04.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.