Giter VIP home page Giter VIP logo

luizfelippesr / magnetizer Goto Github PK

View Code? Open in Web Editor NEW
6.0 3.0 0.0 8 MB

Magnetizer - A Fortran code that post-processes the output of a semi-analytic model of galaxy formation and produces a catalogue of galaxies with detailed radial dependent ISM properties, including magnetic fields

License: GNU General Public License v3.0

Makefile 1.62% Python 30.16% Fortran 68.22%
astrophysics mean-field-dynamo galaxies interstellar-medium

magnetizer's People

Stargazers

avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

magnetizer's Issues

Unable to create output file in parallel

When the magnetizer runs in parallel, it is unable to create an output file.

Instead, the first time IO_flush is called, the code exits with an error:

HDF5-DIAG: Error detected in HDF5 (1.8.16) MPI-process 0:
  #000: ../../../src/H5F.c line 727 in H5Fflush(): unable to flush mounted file hierarchy
    major: File accessibilty
    minor: Unable to flush data from cache
  #001: ../../../src/H5Fmount.c line 699 in H5F_flush_mounts(): unable to flush mounted file hierarchy
    major: File accessibilty
    minor: Unable to flush data from cache
  #002: ../../../src/H5Fmount.c line 660 in H5F_flush_mounts_recurse(): unable to flush file's cached information
    major: File accessibilty
    minor: Unable to flush data from cache
  #003: ../../../src/H5Fint.c line 1185 in H5F_flush(): low level truncate failed
    major: File accessibilty
    minor: Write failed
  #004: ../../../src/H5FD.c line 1909 in H5FD_truncate(): driver truncate request failed
    major: Virtual File Layer
    minor: Can't update object
  #005: ../../../src/H5FDmpio.c line 1985 in H5FD_mpio_truncate(): MPI_File_set_size failed
    major: Internal error (too specific to document in detail)
    minor: Some MPI function failed
  #006: ../../../src/H5FDmpio.c line 1985 in H5FD_mpio_truncate(): MPI_ERR_ARG: invalid argument of some other kind
    major: Internal error (too specific to document in detail)
    minor: MPI Error String
HDF5-DIAG: Error detected in HDF5 (1.8.16) MPI-process 5:
  #000: ../../../src/H5F.c line 727 in H5Fflush(): unable to flush mounted file hierarchy
    major: File accessibilty
    minor: Unable to flush data from cache
  #001: ../../../src/H5Fmount.c line 699 in H5F_flush_mounts(): unable to flush mounted file hierarchy
    major: File accessibilty
    minor: Unable to flush data from cache
  #002: ../../../src/H5Fmount.c line 660 in H5F_flush_mounts_recurse(): unable to flush file's cached information
    major: File accessibilty
    minor: Unable to flush data from cache
  #003: ../../../src/H5Fint.c line 1185 in H5F_flush(): low level truncate failed
    major: File accessibilty
    minor: Write failed
 0:  Error: IO - Error when calling HDF5.  
  #004: ../../../src/H5FD.c line 1909 in H5FD_truncate(): driver truncate request failed
    major: Virtual File Layer
    minor: Can't update object
  #005: ../../../src/H5FDmpio.c line 1985 in H5FD_mpio_truncate(): MPI_File_set_size failed
    major: Internal error (too specific to document in detail)
    minor: Some MPI function failed
  #006: ../../../src/H5FDmpio.c line 1985 in H5FD_mpio_truncate(): MPI_ERR_ARG: invalid argument of some other kind
    major: Internal error (too specific to document in detail)
    minor: MPI Error String

There is a very easy workaround: simply running the code for a single galaxy, on a single processor, beforehand. This will allow the code to create the output file and exit.

If there is pre-existing output file on disk, everything works normally.

Difficulties running using MPI in some systems

@charles-jose has been encountering errors when trying to run the main program using MPI (with mpirun). Running the code in serial mode (i.e. without mpirun) works fine.

Part of the output that @charles-jose gets is shown below.

   |  \/  | __ _  __ _ _ __   ___| |_(_)_______ _ __  
   | |\/| |/ _` |/ _` | '_ \ / _ \ __| |_  / _ \ '__| 
   | |  | | (_| | (_| | | | |  __/ |_| |/ /  __/ |    
   |_|  |_|\__,_|\__, |_| |_|\___|\__|_/___\___|_|    
                |___/      
   Runnning on 36 processors
   Using global parameters file: example/example_global_parameters.in  
   Warm up run  
 0:  Galaxy 1 - Finished after  .704561 s  CPU time
   Warm up done  
   Total number of cycles 1 
   Cycle 1 
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) MPI-process 1:
  #000: ../../../src/H5F.c line 579 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: ../../../src/H5Fint.c line 1168 in H5F_open(): unable to lock the file or initialize file structure
    major: File accessibilty
    minor: Unable to open file
  #002: ../../../src/H5FD.c line 1821 in H5FD_lock(): driver lock request failed
    major: Virtual File Layer
    minor: Can't update object
  #003: ../../../src/H5FDsec2.c line 939 in H5FD_sec2_lock(): unable to flock file, errno = 11, error message = 'Resource temporarily unavailable'
    major: File accessibilty
    minor: Bad file ID accessed
HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) MPI-process 3:
  #000: ../../../src/H5F.c line 579 in H5Fopen(): unable to open file
    major: File accessibilty
    minor: Unable to open file
  #001: ../../../src/H5Fint.c line 1168 in H5F_open(): unable to
[....]

Updating examples

The current examples directory is probably suffering from bit rot. It should be tested and, if needed, replace with a new example input parameters (which should be conveniently small and contains an sufficiently interesting set of galaxies).

Tutorial notebook

It would be useful to have jupyter notebook tutorial which exemplifies the basic use of the code and the python helpers.

Here is the plan for this. I will write two tutorials:

  • Magnetizer main tutorial
    • Compilation
    • Input file preparation
    • Running the Magnetizer program
    • Running using MPI
    • Parameters
    • Input/output files structure
    • MagnetizerRun object basic usage
    • "Extra quantities"
    • Caching
    • Visualization scripts
    • Exporting pandas.DataFrame objects
  • Observables tutorial
    • Line-of-sight integration principles
    • Running the Observables_single program
    • LoS_integrate python helper
    • Running the Observables
    • Looking at the output using MagnetizerRun

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.