Giter VIP home page Giter VIP logo

rhysu / esio Goto Github PK

View Code? Open in Web Editor NEW
11.0 7.0 2.0 2.74 MB

The ExaScale IO (ESIO) library provides simple, high throughput input and output of structured data sets using parallel HDF5. ESIO is designed to support reading and writing turbulence simulation restart files within C, C++, and modern Fortran applications.

Home Page: https://rhysu.github.io/ESIO/

License: GNU Lesser General Public License v2.1

C 48.95% Fortran 4.82% Shell 11.75% Perl 10.50% C++ 3.49% Makefile 4.95% HTML 0.01% M4 15.54%

esio's Introduction

esio's People

Contributors

dmcdougall avatar koomie avatar nicholasmalaya avatar rhysu avatar roystgnr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

esio's Issues

Documentation Dead Link

Hey!

I'm using your libraries as a dependency for another program. I wanted to extend the functionality of the host program, and in doing so, I'm looking to understand the API for your code. Unfortunately, all of the documentation links go to a dead UTexas site. Is there any way to provide the documentation on a new site or via github?!

Thanks a lot!

Compile error on multiple machines

I get a bunch of errors similar to the lines below when compiling esio with gcc 6 or 7. I'm not exactly sure what's causing the error, if it is because of the gcc version or how it can be fixed. Any ideas? I've also attached my config.log and make error output if that helps.

layout.c:170:20: error: conflicting types for 'esio_field_layout2_field_reader'
 #define METHODNAME esio_field_layout2_field_reader

Compiler dependency

@dmcdougall

I tried to install ESIO-0.1.on Wrangler, TACC.

First I configured with "CC=mpicc FC=mpif90 CXX=mpicxx". It configured okay, but showed error when compiling the code with "MAKE".
Next, I configured with "CC=icc FC=ifort CXX=icpc", and it worked just fine.

More specific information is as follows.


~$ module list

Currently Loaded Modules:

  1. TACC-paths 2) Linux 3) cluster-paths 4) intel/15.0.3 5) mvapich2/2.1 6) cluster 7) TACC 8) phdf5/1.8.14

~$ ../configure --prefix=(some_local_dir) --with-hdf5=$TACC_HDF5_BIN/h5pcc CFLAGS="-fPIC" CPPFLAGS="-fPIC" CXXFLAGS="-fPIC" FCFLAGS="-fPIC" LDFLAGS="-L/usr/lib64" CC=mpicc FC=mpif90 CXX=mpicxx
..................long details.............
----------------------------------- SUMMARY -----------------------------------

Package version............... : esio-0.1.9

C++ compiler.................. : mpicxx
C++ compiler flags............ : -fPIC -g -Wno-unknown-pragmas
C compiler.................... : mpicc
C compiler flags.............. : -fPIC -g -traceback -Wall -std=gnu99 -Wcheck -Wpointer-arith -wd424 -wd981 -Wno-unknown-pragmas
Fortran compiler ............. : mpif90
Fortran compiler flags........ : -fPIC -g -warn all -traceback
Install dir................... : /home/01545/pro0104/SOFTWARE/ESIO-0.1.9
HDF CPP flags................. :
HDF C flags................. :
HDF LDFLAGS flags............. :
Build user.................... : pro0104
Build host.................... : login1.wrangler.tacc.utexas.edu
Configure date................ : 2017-08-15 14:40
Build architecture............ : x86_64-unknown-linux-gnu
SVN revision number........... :


~$ make
..............long compilation lines..............................
libtool: link: mpif90 -I.. -I../.. -I../esio -fPIC -g -warn all -traceback -Wl,--as-needed -o .libs/concepts2 concepts2.o -Wl,-rpath -Wl,/opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib -L/usr/lib64 ../esio/.libs/libesiof.so -L/opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib /opt/apps/intel15/mvapich2/2.1/lib/libmpifort.so -L/usr/lib64/ -L/lib /opt/apps/intel15/mvapich2/2.1/lib/libmpi.so -lxml2 -libmad -libumad -libverbs -lrt /opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib/libhdf5_hl.so /opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib/libhdf5.so /opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib/libsz.so -lz -ldl -lm -Wl,-rpath -Wl,/home/01545/pro0104/SOFTWARE/ESIO-0.1.9/lib -Wl,-rpath -Wl,/opt/apps/intel15/mvapich2/2.1/lib -Wl,-rpath -Wl,/opt/apps/intel15/mvapich2_2_1/phdf5/1.8.14/x86_64/lib
ld: cannot find -libverbs
make[3]: *** [concepts2] Error 1
make[3]: Leaving directory /home/01545/pro0104/SOFTWARE/esio-0.1.9/build/examples' make[2]: *** [all] Error 2 make[2]: Leaving directory /home/01545/pro0104/SOFTWARE/esio-0.1.9/build/examples'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/01545/pro0104/SOFTWARE/esio-0.1.9/build'
make: *** [all] Error 2


Issue Writing Large Files

I have encountered a problem writing large array using a single processor. I'm running a serial code that's working with a large array. When I work with smaller arrays (64x64x64, for example) the following example works fine. My *.h5 files contain 1e-6 in every position, like it should. But when I bump up the size, my *.h5 file output just contains 0's.

Here's my minimal working example:

program ESIO_test
    use, intrinsic :: iso_c_binding
    use mpi
    use esio
    implicit none

    integer :: myrank, nprocs, ierr
    real(C_DOUBLE) :: Udata(2,1024,1024,512,3)
    type(esio_handle) :: h

    call mpi_init(ierr)
    call mpi_comm_rank(MPI_COMM_WORLD, myrank, ierr)
    call mpi_comm_size(MPI_COMM_WORLD, nprocs, ierr)

    call esio_handle_initialize(h, MPI_COMM_WORLD)
    call esio_file_create(h,"/work/04114/clarkp/lonestar/fields/512/PS/restart00000000.h5",.true.)

    Udata = 1e-6
    call esio_field_establish(h, 1024, 1, 1024, 1024, 1, 1024, 512, 1, 512, ierr)
    call esio_field_writev_double(h, "u", Udata(:,:,:,:,1), 2)
    call esio_field_writev_double(h, "v", Udata(:,:,:,:,2), 2)
    call esio_field_writev_double(h, "w", Udata(:,:,:,:,3), 2)
    call mpi_barrier(MPI_COMM_WORLD, ierr)

    call esio_file_close(h)
    call esio_handle_finalize(h)
    
    call mpi_finalize(ierr)

end program ESIO_test

I also modified this example to check the "ierr" flags at each step, but they remained 0.

Yes, I know that ESIO is really meant for parallel reading/writing and yes, I know that parallelizing the rest of my code would fix the problem. But up until now, the serial portion of the code has worked fine, and even with large arrays it only takes a minute to run. While switching to generic hdf5 library calls and/or making the code parallel might be better, both would require time to rewrite code and/or extra code complexity. I'd prefer to use a serial code if I can get away with it.

System Information:
My compiler is ifort (IFORT) 16.0.1 20151021 and I'm using the -fopenmp flag.
I'm using the release branch 0.1.9 of ESIO.
I'm using Cray mpich 7.3.0
I'm running this on an interactive session on Lonestar 5 at TACC, with 1 node and 16 tasks allocated. I'm only running the above example with 1 MPI task.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.