bccp / conda-channel-bccp Goto Github PK

View Code? Open in Web Editor NEW

0.0 7.0 1.0 358 KB

building bccp-related packages on the anaconda/bccp channel

Shell 46.51% C++ 8.85% Python 33.38% Batchfile 0.74% C 2.74% Fortran 6.92% Makefile 0.87%

conda-channel-bccp's People

Contributors

Watchers

Forkers

h3jia

conda-channel-bccp's Issues

Python 3.7

This is a placeholder for testing out Python 3.7 support.

Need to survey if any of the packages here do not support 3.7.

Build Python 3.9 binaries.

[ ] pc variants (py38 branch)
[ ] Cori variants.

classylss on mac fails at run time, likely due to -D_FORTIFY_SOURCE=1 or 2 with the new toolschain.

[Traviss-Mac-1345:81272] [ 0] 2 libsystem_platform.dylib 0x00007fff8986252a _sigtramp + 26

[Traviss-Mac-1345:81272] [ 1] 3 libsystem_malloc.dylib 0x00007fff99ce50cc malloc + 42

[Traviss-Mac-1345:81272] [ 2] 4 libsystem_c.dylib 0x00007fff955d36df abort + 129

[Traviss-Mac-1345:81272] [ 3] 5 libsystem_c.dylib 0x00007fff955d3856 _UTF2_init + 0

[Traviss-Mac-1345:81272] [ 4] 6 libsystem_c.dylib 0x00007fff955f9a0c __chk_fail_overlap + 0

[Traviss-Mac-1345:81272] [ 5] 7 libsystem_c.dylib 0x00007fff955f99dc __chk_fail + 0

[Traviss-Mac-1345:81272] [ 6] 8 libsystem_c.dylib 0x00007fff955f9c29 __memccpy_chk + 0

[Traviss-Mac-1345:81272] [ 7] 9 binding.cpython-35m-darwin.so 0x000000150becaa5b input_read_parameters + 60107

pkg-config 0.29.2 incompatible with prgenv_gnu

Anaconda only provides 0.29.2 pkg-config, which is incompatible with prgenv_gnu.

The healpy recipe uses pkg-config as a build dependency, and thus fails to build on Cray.

We need to find a workaround, e.g. build our pkg-config 0.28.0 on bccp. (It is only a build dependency, so it won't be too bad).

Switch Corrfunc back to main repo instead of forked version

The DD(s,mu) functionality has been merged into the master branch now and a new release tagged so we can safely build from the main repo instead of my forked version going forward.

openmpi image not found issue

openmpi image not found issue has plagued to here as well. some mentioned we can avoid this by switching to mpich on both platforms, which seems to be reasonable.

I shall look into this when I get to touch a mac.

Never dropping the package build number.

I think we shall not drop the package build number.
Because travis may have already built a newer version with the higher build number before we drop it when a new version is release. As an example, we see a bunch of 0.2.6 version of nbodykit -- the _3 ones all contained bad dependency, but the fixed version is only in _1 and _2; so they are never installed..

Be ready for Anaconda 5.0

[ ] update mpich2 on Anaconda (perhaps to mpich3)
[ ] update openmpi on Anaconda (for OSX)
[ ] update scripts to use compiler('c'), compiler('fortran'), compiler('cxx').
[ ] replace conda-build-all with some bash voodoo, or fix it; there is no timeline to get it working with conda-build-3.

Check the extension flags to make sure they are 'portable'

See the comment below:

ContinuumIO/anaconda-recipes#129 (comment)

mpsort and pfft-python still uses platform compiler on osx

The cross compilation compiler cannot find mpi.h. It should be related to the fact that openmpi is not properly built with the cross compilation chain, and the env variable override is not working correctly.

Update to include numpy 1.12.x

Somehow it refuses to rebuild binaries for newer version of numpy.

classylss >=v0.2.6 depends on libgfortran from "anaconda" channel

The most recent versions of classylss have a dependency of libgfortran >=3.0.1, but the latest available in the "defaults" channel is libgfortran 3.0.0, while 3.0.1 is available via the "anaconda" channel. I am not sure what the difference is between those. The consequence is that v0.2.5 gets installed by default.

Adding nightly rebuild triggers

We can have a cron job on nersc that triggers a rebuild every night via travis api.

https://docs.travis-ci.com/user/triggering-builds

binary packages do not work on GLIBC_2.14

This is a known issue for builds from Travis. The OS on Travis is slightly newer than most super computing centres, making this a concern.

For example, when I installed bigfile on BlueWaters the binary didn't work with error

(root) yfeng1@h2ologin2:~/BWSim-PFFT/etc> !hdf2
hdf2bigfile 
Traceback (most recent call last):
  File "/u/sciteam/yfeng1/anaconda3/bin/hdf2bigfile", line 4, in <module>
    import bigfile
  File "/u/sciteam/yfeng1/anaconda3/lib/python3.5/site-packages/bigfile/__init__.py", line 3, in <module>
    from .pyxbigfile import BigFileError, BigFileClosedError, BigBlockClosedError
ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /u/sciteam/yfeng1/anaconda3/lib/python3.5/site-packages/bigfile/pyxbigfile.cpython-35m-x86_64-linux-gnu.so)

If this happens, we advice the users revert to use pip install.

The issue is workedaround at NERSC because at NERSC we build nightlies on the NERSC systems.

Switch to mpich-3.x

The package name will be mpich. But the version will be 3.x. We will need these to provide MPI_Iallreduce support in MP-Sort. Also may potentially allow using these linux binaries with shifter on NERSC.

pfft-python not building on Cori

It look pfft is failing to build, but I am not sure why. The recipes for assert.lo, align.lo, alloc.lo, and awake.lo fail. See the build log below.

Any ideas, @rainwoodman?

configure: ****************************************************************
configure: *      Configuring in common/pfft                              *
configure: ****************************************************************
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for style of include used by make... GNU
checking for gcc... /opt/cray/pe/craype/2.5.12/bin/cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /opt/cray/pe/craype/2.5.12/bin/cc accepts -g... yes
checking for /opt/cray/pe/craype/2.5.12/bin/cc option to accept ISO C89... none needed
checking whether /opt/cray/pe/craype/2.5.12/bin/cc understands -c and -o together... yes
checking dependency style of /opt/cray/pe/craype/2.5.12/bin/cc... gcc3
checking for function MPI_Init... yes
checking for mpi.h... yes
checking for /opt/cray/pe/craype/2.5.12/bin/cc option to accept ISO C99... none needed
checking for inline... inline
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by /opt/cray/pe/craype/2.5.12/bin/cc... /usr/common/software/altd/2.0/bin/ld
checking if the linker (/usr/common/software/altd/2.0/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/common/software/altd/2.0/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from /opt/cray/pe/craype/2.5.12/bin/cc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... /opt/cray/pe/craype/2.5.12/bin/cc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if /opt/cray/pe/craype/2.5.12/bin/cc supports -fno-rtti -fno-exceptions... no
checking for /opt/cray/pe/craype/2.5.12/bin/cc option to produce PIC... -fPIC -DPIC
checking if /opt/cray/pe/craype/2.5.12/bin/cc PIC flag -fPIC -DPIC works... yes
checking if /opt/cray/pe/craype/2.5.12/bin/cc static flag -static works... yes
checking if /opt/cray/pe/craype/2.5.12/bin/cc supports -c -o file.o... yes
checking if /opt/cray/pe/craype/2.5.12/bin/cc supports -c -o file.o... (cached) yes
checking whether the /opt/cray/pe/craype/2.5.12/bin/cc linker (/usr/common/software/altd/2.0/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for sincos in -lm... yes
checking for ptrdiff_t... yes
checking for long double... yes
checking size of ptrdiff_t... 8
checking size of MPI_Fint... 4
checking for library containing log... none required
configure: disable build of user manual
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating doc/latexinc.sh
config.status: creating Makefile
config.status: creating pfft.pc
config.status: creating api/pfft.f
config.status: creating api/pfft.f03
config.status: creating api/pfftl.f03
config.status: creating kernel/Makefile
config.status: creating util/Makefile
config.status: creating api/Makefile
config.status: creating gcell/Makefile
config.status: creating tests/Makefile
config.status: creating tests/fortran/Makefile
config.status: creating tests/f03/Makefile
config.status: creating tests/openmp/Makefile
config.status: creating doc/Makefile
config.status: creating config.h
config.status: creating fconfig.h
config.status: linking ../pfft-1.0.8-alpha2-fftw3/tests/build_checks.sh to tests/build_checks.sh
config.status: linking ../pfft-1.0.8-alpha2-fftw3/tests/run_checks.sh to tests/run_checks.sh
config.status: linking ../pfft-1.0.8-alpha2-fftw3/tests/manual_c2c_3d.c to tests/manual_c2c_3d.c
config.status: executing depfiles commands
config.status: executing libtool commands
=== configuring in fftw3 (/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double/fftw3)
configure: running /bin/sh ../../pfft-1.0.8-alpha2-fftw3/fftw3/configure --disable-option-checking '--prefix=/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/build/temp.linux-x86_64-2.7/depends'  '--disable-shared' '--enable-static' '--disable-fortran' '--disable-doc' '--enable-mpi' '--enable-sse2' 'MPICC=/opt/cray/pe/craype/2.5.12/bin/cc' 'CC=/opt/cray/pe/craype/2.5.12/bin/cc' 'CFLAGS=-pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -fvisibility=hidden' --cache-file=/dev/null --srcdir=../../pfft-1.0.8-alpha2-fftw3/fftw3
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for gcc... /opt/cray/pe/craype/2.5.12/bin/cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether /opt/cray/pe/craype/2.5.12/bin/cc accepts -g... yes
checking for /opt/cray/pe/craype/2.5.12/bin/cc option to accept ISO C89... none needed
checking whether /opt/cray/pe/craype/2.5.12/bin/cc understands -c and -o together... yes
checking for style of include used by make... GNU
checking dependency style of /opt/cray/pe/craype/2.5.12/bin/cc... gcc3
checking for C compiler vendor... gnu
checking for /opt/cray/pe/craype/2.5.12/bin/cc option to accept ISO C99... none needed
checking for /opt/cray/pe/craype/2.5.12/bin/cc option to accept ISO Standard C... (cached) none needed
checking whether ln -s works... yes
checking whether make sets $(MAKE)... (cached) yes
checking how to print strings... printf
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by /opt/cray/pe/craype/2.5.12/bin/cc... /usr/common/software/altd/2.0/bin/ld
checking if the linker (/usr/common/software/altd/2.0/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking the maximum length of command line arguments... 1572864
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/common/software/altd/2.0/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... dlltool
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @FILE support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from /opt/cray/pe/craype/2.5.12/bin/cc object... ok
checking for sysroot... no
checking for a working dd... /usr/bin/dd
checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... /opt/cray/pe/craype/2.5.12/bin/cc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if /opt/cray/pe/craype/2.5.12/bin/cc supports -fno-rtti -fno-exceptions... no
checking for /opt/cray/pe/craype/2.5.12/bin/cc option to produce PIC... -fPIC -DPIC
checking if /opt/cray/pe/craype/2.5.12/bin/cc PIC flag -fPIC -DPIC works... yes
checking if /opt/cray/pe/craype/2.5.12/bin/cc static flag -static works... yes
checking if /opt/cray/pe/craype/2.5.12/bin/cc supports -c -o file.o... yes
checking if /opt/cray/pe/craype/2.5.12/bin/cc supports -c -o file.o... (cached) yes
checking whether the /opt/cray/pe/craype/2.5.12/bin/cc linker (/usr/common/software/altd/2.0/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... no
checking whether to build static libraries... yes
checking for ocamlbuild... no
checking for mpicc... /opt/cray/pe/craype/2.5.12/bin/cc
checking for MPI_Init... yes
checking for mpi.h... yes
checking for mpirun... no
checking size of MPI_Fint... 4
checking whether C compiler accepts -msse2... yes
checking for ANSI C header files... (cached) yes
checking libintl.h usability... yes
checking libintl.h presence... yes
checking for libintl.h... yes
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking stddef.h usability... yes
checking stddef.h presence... yes
checking for stddef.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for strings.h... (cached) yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking c_asm.h usability... no
checking c_asm.h presence... no
checking for c_asm.h... no
checking intrinsics.h usability... no
checking intrinsics.h presence... no
checking for intrinsics.h... no
checking for stdint.h... (cached) yes
checking mach/mach_time.h usability... no
checking mach/mach_time.h presence... no
checking for mach/mach_time.h... no
checking sys/sysctl.h usability... yes
checking sys/sysctl.h presence... yes
checking for sys/sysctl.h... yes
checking altivec.h usability... no
checking altivec.h presence... no
checking for altivec.h... no
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking for size_t... yes
checking whether time.h and sys/time.h may both be included... yes
checking for long double... yes
checking for hrtime_t... no
checking size of int... 4
checking size of unsigned int... 4
checking size of long... 8
checking size of unsigned long... 8
checking size of long long... 8
checking size of unsigned long long... 8
checking size of size_t... 8
checking size of ptrdiff_t... 8
checking for uintptr_t... yes
checking size of float... 4
checking size of double... 8
checking size of fftw_r2r_kind... 4
checking for working alloca.h... yes
checking for alloca... yes
checking for working strtod... yes
checking for vprintf... yes
checking for _doprnt... no
checking for sin in -lm... yes
checking for BSDgettimeofday... no
checking for gettimeofday... yes
checking for gethrtime... no
checking for read_real_time... no
checking for time_base_to_time... no
checking for drand48... yes
checking for sqrt... yes
checking for memset... yes
checking for posix_memalign... yes
checking for memalign... yes
checking for _mm_malloc... no
checking for _mm_free... no
checking for clock_gettime... yes
checking for mach_absolute_time... no
checking for sysctl... yes
checking for abort... yes
checking for sinl... yes
checking for cosl... yes
checking for snprintf... yes
checking whether sinl is declared... yes
checking whether cosl is declared... yes
checking whether sinq is declared... no
checking whether cosq is declared... no
checking whether memalign is declared... yes
checking whether drand48 is declared... yes
checking whether srand48 is declared... yes
checking whether posix_memalign is declared... yes
checking for _rtc intrinsic... no
checking for isnan... yes
checking whether C compiler accepts -mpreferred-stack-boundary=4... yes
checking whether the stack is at least 8-byte aligned by gcc... checking whether C compiler accepts -malign-double... yes
yes
checking whether a cycle counter is available... yes
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating support/Makefile
config.status: creating genfft/Makefile
config.status: creating kernel/Makefile
config.status: creating simd-support/Makefile
config.status: creating dft/Makefile
config.status: creating dft/scalar/Makefile
config.status: creating dft/scalar/codelets/Makefile
config.status: creating dft/simd/Makefile
config.status: creating dft/simd/common/Makefile
config.status: creating dft/simd/sse2/Makefile
config.status: creating dft/simd/avx/Makefile
config.status: creating dft/simd/altivec/Makefile
config.status: creating dft/simd/neon/Makefile
config.status: creating rdft/Makefile
config.status: creating rdft/scalar/Makefile
config.status: creating rdft/scalar/r2cf/Makefile
config.status: creating rdft/scalar/r2cb/Makefile
config.status: creating rdft/scalar/r2r/Makefile
config.status: creating rdft/simd/Makefile
config.status: creating rdft/simd/common/Makefile
config.status: creating rdft/simd/sse2/Makefile
config.status: creating rdft/simd/avx/Makefile
config.status: creating rdft/simd/altivec/Makefile
config.status: creating rdft/simd/neon/Makefile
config.status: creating reodft/Makefile
config.status: creating threads/Makefile
config.status: creating api/Makefile
config.status: creating mpi/Makefile
config.status: creating libbench2/Makefile
config.status: creating tests/Makefile
config.status: creating doc/Makefile
config.status: creating doc/FAQ/Makefile
config.status: creating tools/Makefile
config.status: creating tools/fftw_wisdom.1
config.status: creating tools/fftw-wisdom-to-conf
config.status: creating m4/Makefile
config.status: creating fftw.pc
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
make  all-recursive
make[1]: Entering directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double'
Making all in fftw3
make[2]: Entering directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double/fftw3'
make  all-recursive
make[3]: Entering directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double/fftw3'
Making all in support
make[4]: Entering directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double/fftw3/support'
make[4]: Nothing to be done for 'all'.
make[4]: Leaving directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double/fftw3/support'
Making all in kernel
make[4]: Entering directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double/fftw3/kernel'
/bin/sh ../libtool  --tag=CC   --mode=compile /opt/cray/pe/craype/2.5.12/bin/cc -DHAVE_CONFIG_H -I. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel -I..  -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/simd   -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -fvisibility=hidden -MT align.lo -MD -MP -MF .deps/align.Tpo -c -o align.lo ../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel/align.c
/bin/sh ../libtool  --tag=CC   --mode=compile /opt/cray/pe/craype/2.5.12/bin/cc -DHAVE_CONFIG_H -I. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel -I..  -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/simd   -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -fvisibility=hidden -MT alloc.lo -MD -MP -MF .deps/alloc.Tpo -c -o alloc.lo ../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel/alloc.c
/bin/sh ../libtool  --tag=CC   --mode=compile /opt/cray/pe/craype/2.5.12/bin/cc -DHAVE_CONFIG_H -I. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel -I..  -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/simd   -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -fvisibility=hidden -MT assert.lo -MD -MP -MF .deps/assert.Tpo -c -o assert.lo ../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel/assert.c
/bin/sh ../libtool  --tag=CC   --mode=compile /opt/cray/pe/craype/2.5.12/bin/cc -DHAVE_CONFIG_H -I. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel -I..  -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/simd   -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -fvisibility=hidden -MT awake.lo -MD -MP -MF .deps/awake.Tpo -c -o awake.lo ../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel/awake.c
libtool: compile:  /opt/cray/pe/craype/2.5.12/bin/cc -DHAVE_CONFIG_H -I. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel -I.. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/simd -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -fvisibility=hidden -MT alloc.lo -MD -MP -MF .deps/alloc.Tpo -c ../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel/alloc.c -o alloc.o
libtool: compile:  /opt/cray/pe/craype/2.5.12/bin/cc -DHAVE_CONFIG_H -I. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel -I.. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/simd -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -fvisibility=hidden -MT align.lo -MD -MP -MF .deps/align.Tpo -c ../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel/align.c -o align.o
libtool: compile:  /opt/cray/pe/craype/2.5.12/bin/cc -DHAVE_CONFIG_H -I. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel -I.. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/simd -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -fvisibility=hidden -MT awake.lo -MD -MP -MF .deps/awake.Tpo -c ../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel/awake.c -o awake.o
libtool: compile:  /opt/cray/pe/craype/2.5.12/bin/cc -DHAVE_CONFIG_H -I. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel -I.. -I../../../pfft-1.0.8-alpha2-fftw3/fftw3/simd -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -fvisibility=hidden -MT assert.lo -MD -MP -MF .deps/assert.Tpo -c ../../../pfft-1.0.8-alpha2-fftw3/fftw3/kernel/assert.c -o assert.o
Makefile:459: recipe for target 'assert.lo' failed
Makefile:459: recipe for target 'align.lo' failed
Makefile:459: recipe for target 'alloc.lo' failed
Makefile:459: recipe for target 'awake.lo' failed
make[4]: Leaving directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double/fftw3/kernel'
Makefile:642: recipe for target 'all-recursive' failed
make[3]: Leaving directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double/fftw3'
Makefile:507: recipe for target 'all' failed
make[2]: Leaving directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double/fftw3'
Makefile:615: recipe for target 'all-recursive' failed
make[1]: Leaving directory '/global/common/cori/contrib/bccp/anaconda3/conda-bld/pfft-python_1502646269559/work/tmp-pfft-1.0.8-alpha2-fftw3/double'
Makefile:480: recipe for target 'all' failed

classylss failing to build on Cori

The error message is:

g++ -pthread -shared -L/usr/common/contrib/bccp/anaconda3/conda-bld/classylss_1502645335512/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib -Wl,-rpath=/usr/common/contrib/bccp/anaconda3/conda-bld/classylss_1502645335512/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib,--no-as-needed build/temp.linux-x86_64-2.7/classylss/binding.o -L/usr/common/contrib/bccp/anaconda3/conda-bld/classylss_1502645335512/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib -Lbuild/temp.linux-x86_64-2.7/lib -Lbuild/temp.linux-x86_64-2.7 -lclass -lgomp -lgfortran -lpython2.7 -o build/lib.linux-x86_64-2.7/classylss/binding.so -g -fPIC
/usr/bin/ld: /usr/lib64/libmvec_nonshared.a(svml_finite_alias.oS): relocation R_X86_64_PC32 against undefined symbol `_ZGVbN2v_log@@GLIBC_2.22' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

This seems to be due to this bug in GLIBC 2.22: https://sourceware.org/bugzilla/show_bug.cgi?id=19590

Although it looks Edison also is using GLIBC 2.22 and this builds fine on Edison. Any ideas, @rainwoodman ?

Add pmesh abopt fastpm-python and nbodykit

pmesh
abopt
nbodykit
fastpm-python

Add depencency on mpich or openmpi for bigfile, mpsort and pfft-python.

These packages link directly against MPI --therefore by definition they depend on the mpi library . This may help out identifying which packages needs to be locally recompiled (anyone that directly depend on mpich).

Add cfastpm and mpgadget.

These are pure C packages. And we also need to pin the libc version.

Migrate to github actions.

Travis-CI stopped offering unlimited free time to OSS.

halotools

halotools is not built on bccp, as a result, installing nbodykit from bccp by default only gives us 0.2.0. I propose we include halotools.
@nickhand

Use Circle-CI to build packages on Linux.

I found a docker image that is used to build anaconda. We shall use that one for all of the packages:

https://github.com/ContinuumIO/docker-images/tree/master/conda_builder_linux

This will pinned down the ABI of glibc and libstdc++ to an earlier version. It is still tricky about the C++99 / 89, once we get there.

circle allows any docker image for the build. Travis doesn't.
OSX on travis is free, on circle it is not.
Once the circle is working I'll find a way to refactor the build script such that it works for both travis and circle.

fitsio cannot find libbz2.so on NERSC jupyterhub

The reason is conda only packages libbz2.a. The compilation is done on a system with libbz2.so, so the compiler picked up libbz2.so instead of using libbz2.a staticly.

I do not know of a way of easily fixing this. Current workaround is to reinstall fitsio with pip install -U --force --no-deps --no-cache fitsio.

I will file a bug against conda about libbz2 not packaging the .so file.

Automatically fetch the latest version from pypi.

This way we will omit the version numbers in requirements.yml.

Then we can have a cron job on nersc that triggers a rebuild every night via travis api.

https://docs.travis-ci.com/user/triggering-builds

Improve interpolation with conda-forge.

It appears that the mpich package on conda-forge is finally compatible with the new anaconda compiling toolchain. We shall rebase our binaries on top of conda-forge.

This will also allow us to get rid of a few other 'platform' packages, e.g. mpi4py, healpy (which has been causing problems). Also means we will be able to pullin their pytorch.

However we do not want to port nbodykit their, since the NERSC scripts still use the recipes here for building. In principle though we can fetch conda-forge recipes and use those at nersc; but that's less urgent.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.