xcompact3d / x3div Goto Github PK
View Code? Open in Web Editor NEWXcompact3d miniapp
License: BSD 3-Clause "New" or "Revised" License
Xcompact3d miniapp
License: BSD 3-Clause "New" or "Revised" License
TARGET: 04/02/2022
The aim is to be able to test automatically derivatives/interpolations/filters regularly (especially after an update of the code)
-UT for each first derivative subroutine in x y and z directions
-UT for each second derivative subroutine in x y and z directions
-UT for each interpolation subroutine in x y and z directions
-UT for each filter subroutine in x y and z directions
-UT for the Poisson solver
Each UT will have to work for all possible sets of boundary conditions. The idea is to define a cos or a sin function to be derived/interpolated, and compare the results with the analytical solution. An error threshold would have to be defined for automatic testing.
See other issue raised by Thibault for more information.
compilation of data for the TGV case on CPUs/GPUs for each version of x3div (when TGV capability is available). The aim is to produce strong and weak scaling plots for various spatial resolution (from 65^3 to 2049^3) with as much compute nodes as possible.
Data should be collected in a way that they can be reproducible (compiler version, mpi version, github version, etc.)
Data should first be collected for ARCHER2.
One issue is to agree on a way to measure the performance (current MPI tools might not be the correct way of doing this).
To be discussed: how and where to store the data on Github
Following commit ef6dfb9, explicit openacc directive allow the generic fft to run on GPU. Do concurrent should be used instead of explicit openacc directives.
-->Testing with TGV case with 128^3, 256^3, 512^3 and 1024^3 mesh nodes.
--> Properly document the benchmarking for reproducibility (machine, compiler, option, github version, etc.)
TARGET: End-Feb
Currently, compilation of the code can be performed using the provided
On the one hand, keeping the Makefile is probably a good idea so that one (not very familiar with the cmake syntax) can quickly test the code on new architectures. On the other hand, keeping both the cmake and autotools is probably a bad idea, and the autotools should be removed later on (v3).
It could be interesting to have a Makefile with explicit file dependencies so that parallel make would work with both Makefile and cmake.
The compilation using cmake should be clearly documented. Some clusters have support for cmake but no support for ccmake or cmake-gui. Thus, it is important to document clearly all the cmake options one can use from the command-line.
To make sure that x3div can use the generic FFT (no need for external librairies) on CPU/GPU, and the cuFFT on NVIDIA GPUs. Default setting should be FFT generic option.
It should be noted that the generic FFT currently need explicit Openacc code to run on NVIDIA GPUs. We need an update on this from NVIDIA. See other issue raised for this problem with openacc (we want to avoid openacc and rely only on do concurrent.
Discussion about FFTW: do we need to have this option available?
-functions for MPI (brainstorming)
Continue to develop a Cmake framework for X3div, with the possibility to use Fortran compilers on CPUs and GPUs. To start with, the framework should work with GCC, Nvidia and Intel compilers, as well as the compilers available on ARCHER2.
This would help keep associated functionality together, for example "tools" is something of a grabbag - a "restart" submodule could be factored out, the interfaces are defined in the tools module and implemented in the restart submodule. This keeps changes isolated and also benefits users as a small code change in a submodule does not trigger a full recompile
-change the loops with do concurrent
TARGET: next hackathon
GPU version with TGV case implemented
To make a start on using github actions we could have a simple workflow that builds x3div and runs the default benchmark.
This would be run upon pushing new commits to the repo.
A minimum setup could be
CMakeLists.txt
for configuration and particularly interfacing with pFUnit.Currently (17.02.2022) some tests are implemented in the hack_doconcurrent
branch using CTest and pFUnit, so I'm assuming some of you know about this testing framework. pFUnit looks like a popular option for fortran. It's maybe not as well documented and feature-rich as frameworks in other languages, but seems to be used by many projects out there. Alternatives are Funit, FRUIT and Ftunit but pFUnit is the only one that ibviusly looks actively maintained?
I've played around with it, it's straightforward to install and use. At least following basic examples available at
https://github.com/Goddard-Fortran-Ecosystem/pFUnit_demos
It also had support for MPI. I don't know if this is common among fortran frameworks, but it sounds relevant to me.
Tests are written in free form fortran, e.g.
@test
subroutine test_square()
use Square_mod
use funit
@assertEqual(9., square(3.), 'square(3)')
end subroutine test_square
that is meant to be parsed and compiled. But a CMake module is provided that takes care of everything, so that
add_pfunit_ctest (my_tests
TEST_SOURCES ${test_srcs}
LINK_LIBRARIES sut # your application library
)
should be enough to generate test executable my_tests
. See Using pFUnit.
fix the time advancement part of x3div
(we need 2DECOMP&FFT to work as an independent library so it can be maintained properly)
-Github sub-project (Paul), if bad idea then go for external library option
TARGET: Mid-FEB
To simply copy paste the MPI I/O tools from xcompact3d to x3div to allow for post-processing of the TGV case.
I/O tools would have to become "do concurrent" friendly.
It would be good to have ADIOS2 capabilities as well.
Discussion to have with Paul about the new Py4incompact3d which allows for parallel post-processing (only need to save ux, uy, uz and pressure in production runs). Potentially possibility to simplify the post-processing tools when running simulations.
The idea here is to have the 2DECOMP&FFT files in a separate directory as the Xcompact3d files. At the moment, there is a 2decomp directory inside the x3div project.
The aim is to have a dedicated 2DECOMP&FFT project, under the Xcompact3d umbrella.
It will allow users to raise dedicated queries/issues for the 2DECOMP&FFT library and will allow its use outside Xcompact3d.
The library should be compiled independently of Xcompact3d and use as an external library; Makefile and cmake in Xcompact3d will have to be modified accordingly.
IMPORTANT: the 2decomp files in x3div are not up to date so we will need to start with the 2decomp files from incompact3d
The FFT algorithm in fft_generic could use a do concurrent construct here ?
x3div/decomp2d/fft_generic.f90
Lines 81 to 91 in 2421f3c
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.