Hi,

We modified some of the code and it supports GPU acceleration. We are not sure whether this is helpful or not. You could check it out.

https://github.com/eee4017/vpic-GPU

In the WRITE_HEADER_V0 macro below, the values 0xcafe and 0xdeadbeef are assigned to a short int and int respectively:

But the guaranteed minimum upper limits of short int and int are 0x7fff and 0x7fffffff, so the constants could overflow. From what I've seen online, signed integer overflow is undefined behaviour, so could technically give different values with different compilers. Presumably all compilers handle it the same way so it's never been a problem.

Two possible solutions that don't break people's postprocessing tools:

• Leave it as is, because it probably doesn't matter
• Assign the overflowed values instead

Around line 212, the following lines (seem?) to be repeated?:

   pi->i                 = nn - range[face]; 
   pi->sp_id             = sp_id;

Hi,

I tried to run the existing ctests on grizzly and got failed tests. I ran

cmake -DENABLE_INTEGRATED_TESTS=ON -DENABLE_UNIT_TESTS=ON .. followed by make -j 36 (which was surprisingly slow) and then ctest.

pcomm, dump, parallel and test_collision_script failed. The failure for pcomm was

8: *** Advancing 8: Recving particle with global_id 0 on rank 1 8: Recving particle with global_id 0 on rank 3 8: Recving particle with global_id 0 on rank 7 8: [gr1485:16752] *** Process received signal *** 8: [gr1485:16752] Signal: Segmentation fault (11) 8: [gr1485:16752] Signal code: Address not mapped (1) 8: [gr1485:16752] Failing at address: (nil) 8: [gr1485:16754] *** Process received signal *** 8: [gr1485:16754] Signal: Segmentation fault (11) 8: [gr1485:16754] Signal code: Address not mapped (1) 8: [gr1485:16754] Failing at address: (nil) 8: [gr1485:16758] *** Process received signal *** 8: [gr1485:16758] Signal: Segmentation fault (11) 8: [gr1485:16758] Signal code: Address not mapped (1) 8: [gr1485:16758] Failing at address: (nil) 8: [gr1485:16752] [ 0] /lib64/libpthread.so.0(+0xf630)[0x2b38225a9630] 8: [gr1485:16752] [ 1] pcomm(find_species_name+0x16)[0x413816] 8: [gr1485:16752] [ 2] pcomm(_ZN15vpic_simulation16user_diagnosticsEv+0x1f)[0x40a15f] 8: [gr1485:16754] [ 0] /lib64/libpthread.so.0(+0xf630)[0x2b66a4372630] 8: [gr1485:16754] [ 1] [gr1485:16758] [ 0] /lib64/libpthread.so.0(+0xf630)[0x2b596445f630] 8: [gr1485:16758] [ 1] pcomm(find_species_name+0x16)[0x413816] 8: [gr1485:16758] [ 2] pcomm(_ZN15vpic_simulation16user_diagnosticsEv+0x1f)[0x40a15f] 8: [gr1485:16758] [ 3] pcomm(_ZN15vpic_simulation7advanceEv+0xbcf)[0x41ad2f] 8: [gr1485:16758] [ 4] pcomm(main+0x15f)[0x409c3f] 8: [gr1485:16758] [ 5] pcomm(find_species_name+0x16)[0x413816] 8: [gr1485:16754] [ 2] pcomm(_ZN15vpic_simulation16user_diagnosticsEv+0x1f)[0x40a15f] 8: [gr1485:16754] [ 3] pcomm(_ZN15vpic_simulation7advanceEv+0xbcf)[0x41ad2f] 8: [gr1485:16754] [ 4] pcomm(main+0x15f)[0x409c3f] 8: [gr1485:16754] [ 5] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b66a542b545] 8: [gr1485:16754] [ 6] pcomm[0x409a29] 8: [gr1485:16754] *** End of error message *** 8: [gr1485:16752] [ 3] pcomm(_ZN15vpic_simulation7advanceEv+0xbcf)[0x41ad2f] 8: [gr1485:16752] [ 4] pcomm(main+0x15f)[0x409c3f] 8: [gr1485:16752] [ 5] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b3823662545] 8: [gr1485:16752] [ 6] pcomm[0x409a29] 8: [gr1485:16752] *** End of error message *** 8: /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b5965518545] 8: [gr1485:16758] [ 6] pcomm[0x409a29] 8: [gr1485:16758] *** End of error message *** 8: -------------------------------------------------------------------------- 8: mpiexec noticed that process rank 1 with PID 16752 on node gr1485 exited on signal 11 (Segmentation fault).

Could you add some readme notes on how visualization is done? My experience with other simulation tools is that a separate program is used for visualizing the simulation results.

hydro_p_pipeline_v4.cc, hydro_p_pipeline_v8.cc, and hydro_p_pipeline_v16.cc advance particle velocities to t+0.5dt instead of t as in hydro_p_pipeline.cc. The resulting hydro quantities are not at the same time as the fields. Data analysis using both hydro and fields tends to have problems. The additional momentum update after Boris rotation is the problem and should be removed.

ux  += hax;
uy  += hay;
uz  += haz;

Trying to output material_ids is broken

and

assume that all fields of a field_t are 4 bytes wide, but material_id is only 2 bytes. The resulting output is not as expected.

Required Features:

• Add a per-species unique particle ID:
  • Add a way that the overhead/functionality can be disabled at compile time
  • Add a way to only have the global IDs be enabled for certain species
• Have an optional way to list which global_ids in a species are in fact tracers to support dynamic tracers populations ("fast ones", "ones who hit wall X", etc):
  • Have some nice easy user-callable method to populate tracers (every nth, x%)
  • Have some predicate based way to select particles? (Partially implemented in global_particle_id branch)
  • [OPTIONAL] Is there some smart way where we can manage an explicit tracer species, where we collected marked tracers? Maybe we don't need this..
• Allow the user to store custom and configurable data for tracer particles:
  • Add some additional array to store data which is user selectable?
  • Let the user select the type of this array at compile time (default: float)
• Sane and nice IO interface
  • Buffering up multiple timesteps before actually dumping
  • Try minimize the number of off load analysis / processing required
  • Avoid writing a million tiny files
  • HDF5 are first output type, add support for VPIC Binary Format later (similar to existing input-deck tracer functionality)
    • Can we make the IO format line up with the proposed changes in
  • [OPTIONAL] Nice example of users can hunt down tracers which left a given rank

Code Quality:

• Provide example deck that uses the new tracer interface
  • Must have an good simple example of how to iterate over tracers and calculate some derived quantity touching fields (likely as a user_diagnostics)
• Add a unit/"deck" test that uses tracer interface
  • Add a unit test to check if ability to compile tracer in/out works
  • Test if it survives a check point restart

This is obviously not a very likely scenario, as we run on meshes much bigger than this, but it may be a useful case for testing computational intensive modules (and writing unit tests etc).

The crux of the problem seems to come down to a check that, for each dimension, does:

const float px = (nx>1) ? g->rdx : 0;

If n_=1 it will set p_ to be 0. This is then used in a division (alphadt = 0.3888889/( px*px + py*py + pz*pz ); ) so inf/nans propagates throughout anything that touches the fields.

Hi devs：
I am new to using the VPIC for studying collisionless shocks in space physics. After 'configure' and 'make' steps, which are all succeful, an error occurred when submitting the Harris test in sample folder. Can someone provide me some clues to solve the problem?

Thanks a lot!

#-------------------------------------------------------------------------------------#

/usr/bin/c++ -DVPIC_USE_PTHREADS -rdynamic -I. -I/home/ckyu/vpic-master/src -std=c++11 -I/data/software/intel/oneapi/mpi/2021.5.1/include -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /data/software/intel/oneapi/mpi/2021.5.1/lib/release -Xlinker -rpath -Xlinker /data/software/intel/oneapi/mpi/2021.5.1/lib -Xlinker --enable-new-dtags -L/data/software/intel/oneapi/mpi/2021.5.1/lib -g -DVPIC_USE_PTHREADS -DINPUT_DECK=../sample/harris /home/ckyu/vpic-master/deck/main.cc /home/ckyu/vpic-master/deck/wrapper.cc -o harris.Linux -Wl,-rpath,/home/ckyu/vpic-master/ckyu_build -L/home/ckyu/vpic-master/ckyu_build -lvpic /data/software/intel/oneapi/mpi/2021.5.1/lib/libmpicxx.so /data/software/intel/oneapi/mpi/2021.5.1/lib/release/libmpi.so /usr/lib64/librt.so /usr/lib64/libpthread.so /usr/lib64/libdl.so /data/software/intel/oneapi/mpi/2021.5.1/lib/release/libmpi.so /usr/lib64/librt.so /usr/lib64/libpthread.so /usr/lib64/libdl.so -lpthread -ldl

HDF5 attributes writen to the file can be incorrect depending on which process writes last. Take for example writing the field attribute VPIC-ArrayUDF-GEO,

VPIC does (by all the processes):

 float attr_data[2][3];
   attr_data[0][0] = grid->x0;
   attr_data[0][1] = grid->y0;
   attr_data[0][2] = grid->z0;
   attr_data[1][0] = grid->dx;
   attr_data[1][1] = grid->dy;
   attr_data[1][2] = grid->dz;
   hsize_t dims[2];
   dims[0] = 2;
   dims[1] = 3;
   hid_t va_geo_dataspace_id = H5Screate_simple(2, dims, NULL);
   hid_t va_geo_attribute_id = H5Acreate2(file_id, "VPIC-ArrayUDF-GEO", H5T_IEEE_F32BE, va_geo_dataspace_id, H5P_DEFAULT, H5P_DEFAULT);
   H5Awrite(va_geo_attribute_id, H5T_NATIVE_FLOAT, attr_data);

but for the HarrisHDF5 case, grid->y0 can have different values between processes. Hyperslab selection can not be used with attributes, so all the processes write their entire array. Hence, the value that gets written to the attribute data is whichever process writes last.

Hello,

Dear developers, I have two questions:

1- as far as I know the VPIC code is open source so every Body can use it? is it correct.?
2- is there any Manual which I can read to understand how to create an Input file

thanks in advance
masoud

As far as I'm aware, there is no versioning convention used in the VPIC source apart from distinguishing between "v4" and the master branch.

Can a VPIC versioning convention be adopted? The big differences are between v407 and master, but there may also be slight version differences between releases which may result in different output. Not having a versioning convention means an ad-hoc approach to versioning deployments, such as timestamps.

Hi,

This is a new user question: In src/vpic/dump.cc, the band_interleave branch of hydro_dump seems to not correctly set the dim array to include ghost layer:

    dim[0] = nxout;
    dim[1] = nyout;
    dim[2] = nzout;

while the field_dump does

    dim[0] = nxout+2;
    dim[1] = nyout+2;
    dim[2] = nzout+2;

It appears to me that the two extra cells are ghost cells. For direct dump (no stride), I believe the dim array should include them, since the memory block is dumped as a whole.

Thanks,
Liang

PS:

• Commit I am working on (current master as of 11/01/2017): 91a209d
• Link to file: https://github.com/lanl/vpic/blob/91a209d68f8d0fa8b58c7201dd0411a3fcb57e3e/src/vpic/dump.cc

I am trying to improve the performance of move_p, and when I use move_p_v4( ),which is the move_p( ) when V4_ACCELERATION is defined, some data are calculated wrong and after some steps(like 44 steps in my case), NAN shows up in interpolator's data, it influences all data I use and gives the wrong answer in following calculating.
I tried to use gdb to trace the bug, but it doesn't seem to be obvious and it's hard to find out why, what can I do?

In line 224 of compute_div_b_err_pipeline.cc:
It should be for( ; n_voxel > 7; n_voxel-=8) instead of for( ; n_voxel > 3; n_voxel -=8).
Or I just misunderstood this code?

Can you confirm that contributions to the project are licensed inbound to the project under the same license as the outbound license?

thanks

On some platforms/compilers, this will generate a cast of the single precision float variables.

I think the literals should be floats not doubles?

Do field_t and hydro_t need to padded?

https://github.com/lanl/vpic/blob/devel/src/species_advance/standard/pipeline/advance_p_pipeline.cc#L310

Currently only throws a warning about a situation where we are likely allowing the code to go out of bounds in memory. We should either

A) Have this section https://github.com/lanl/vpic/blob/devel/src/species_advance/standard/pipeline/advance_p_pipeline.cc#L230 undo the bit shift done by move_p

or B) throw an error and die

The momentum (ux, uy, uz) needs to be normalized by the particle mass * speed of light, but this does not seem to be documented anywhere

The traditional way to set compiler optimization flags with cmake is to use CMAKE_BUILD_TYPE. The defaults with cmake are something like:

CMAKE_C_FLAGS_DEBUG:STRING=-g
CMAKE_C_FLAGS_MINSIZEREL:STRING=-Os -DNDEBUG
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
CMAKE_C_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG

For portability, the exact flag string used by cmake depends on which compiler it detects (since different compilers may have different flag syntax). The above gcc flags come from Modules/Compiler/GNU.cmake in the cmake library directory.

CMAKE_[lang]FLAGS[build_type] gets merged into CMAKE_[lang]_FLAGS. Note that if CMAKE_BUILD_TYPE is not specified (i.e. empty) then nothing is merged.

If you specify both CMAKE_BUILD_TYPE and put optimization flags in CMAKE_[lang]_FLAGS at the same time, you'll get both sets of flags in your compile command line. Example:

% cat ../CMakeLists.txt
add_executable(testin testin.c)
% 
% cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_FLAGS="-O0" ..
-- The C compiler identification is GNU 5.5.0
-- The CXX compiler identification is GNU 5.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /tmp/testin/b
% 
% make VERBOSE=1 testin.o
make -f CMakeFiles/testin.dir/build.make CMakeFiles/testin.dir/testin.c.o
Building C object CMakeFiles/testin.dir/testin.c.o
/usr/bin/cc   -O0 -O3 -DNDEBUG -o CMakeFiles/testin.dir/testin.c.o   -c /tmp/testin/testin.c
%   

Note that both "-O0" (from CMAKE_C_FLAGS) and "-O3 -DNDEBUG" (from CMAKE_BUILD_TYPE=Release) are both specified on the compile command line.

This explains why "-O3" unexpectedly appears twice in the compile command ("make VERBOSE=1") when you run lanl-cts1 on GR, for example:

[  3%] Building C object CMakeFiles/vpic.dir/src/boundary/link.c.o
/usr/projects/hpcsoft/toss3/grizzly/openmpi/2.1.2-intel-18.0.5/bin/mpicc -DMIN_NP=128 -DUSE_V4_AVX2 -DUSE_V8_AVX2 -DVPIC_USE_LEGACY_SORT -DVPIC_USE_PTHREADS -I/users/ccranor/src/vpic  -g -O3 -inline-forceinline -qoverride-limits -no-ansi-alias -Winline -qopt-report=5 -qopt-report-phase=all -diag-disable 10397 -Wl,--export-dynamic -O3 -DNDEBUG   -std=gnu99 -o CMakeFiles/vpic.dir/src/boundary/link.c.o   -c /users/ccranor/src/vpic/src/boundary/link.c

It would be better to let cmake control the optimization flags using CMAKE_BUILD_TYPE and not add them to CMAKE_[lang]_FLAGS. To override the cmake defaults (if needed), you can reset the build related-variables like CMAKE_CXX_FLAGS_RELEASE.

When restarting in 1.1, global variables are initialised to 0. This affects decks which use the globals (e.g. sample/harris doesn't write output after restarting, sample/asymm4sp crashes due to division by zero when checking restart_interval)

I believe it's related to this change:
#48
Should decks be modified not to use globals or is there a way to store them?

The child langmuir emitter moves newly emitted and 'aged' particles by calling the particle mover as:
pm[nm].disp##X = wu##Xrd##X;
pm[nm].disp##Y = wu##Yrd##Y;
pm[nm].disp##Z = wu##Zrd##Z;
pm[nm].i = np-1;
nm += move_p( p, pm, a, g, qsp ); \

However, if a particle already had a boundary interaction in 'advance_p' it is at the top of the 'pm' list. The call of move_p in the emitter code then attempts to move the already out of bounds particle, instead of the newly emitted particle as intended.

This can be fixed by using a local particle mover, as is used in 'advance_p'. That is:

DECLARE_ALIGNED_ARRAY( particle_mover_t, 16, local_pm, 1 );
...
local_pm->disp##X = wu##Xrd##X;
local_pm->disp##Y = wu##Yrd##Y;
local_pm->disp##Z = wu##Zrd##Z;
local_pm->i = np-1;
if ( move_p( p, local_pm, a, g, qsp ) ) {
pm[nm++] = local_pm[0];
}

Hi
Is there a description of native VPIC format that one can use to read/visualize data from let's say a
home written python script instead of Paraview ?
Thanks in advance
Denis

Hi VPIC maintainers,

In CAMPA, we recently started to create an overview catalogue of Accelerator Simulation Codes. Would you be interested to update the entry for VPIC? :)

• database: https://github.com/campa-consortium/accelerator_simulation_codes
  • update instructions: simply edit codes.json via a pull request
  • keywords: https://github.com/campa-consortium/accelerator_simulation_codes#keywords-schema
• live homepage: https://campa-consortium.github.io/accelerator_simulation_codes/

Thanks,
Axel

Building with -DENABLE_MPI=OFF still results in an MPI build. This is because config/project.cmake forces ENABLE_MPI to be TRUE. Fixing this still results in build errors, since MPI dependencies are included for the compile.

Hi
Is there some VPIC input example describing the interaction of a high-intensity
laser with a solid target ?

The readMe.md only gives the example in the Linux, Is VPIC available in Windows?

src/util/mp/MPWrapper.h is a wrapper around DMPPolicy or RelayPolicy, depending on configuration. RelayPolicy is not a valid build as it is missing its include dependencies (e.g. ConnectionManager.h, P2PConnection.h, checkpt.h).

Hi, I am just a newbie learning how to use this code and I want to be able to introduce electric field into my simulation. In order to do that, I included set_region_field inside begin_initialization and tried to add some magnetic and electric fields. Magnetic fields worked well but there is no electric field (as I see from the Paraview visualizations). Later I tried to understand and use field(ix, iy, iz) (the declaration is at ./src/vpic.h line 290) but I couldn't understand what these parameters meant and how to use them. Any help will be appreciated!

Correction: the declaration seems to be at ./src/field_advance/standard/remote.c line 15

There was a "feature" in old catch in which it emitted some raw ASM into the executable. The previous ASM they generated is not compatible with some modern non x86-platforms but they have fixed that in a new version

It would be neat if you could update the catch version to something more modern :). Should be as simple as just updating the file

Hello,

I could compile the vpic and execute the sample application based on readme, where could I get further information such as how to visualize the simulation data base on paraview and more details about the vpic data?

Thanks a lot for your help!

Can we bring them together some how? parameterize out the paths?

Dear VPIC Community,

Vpic seems to me that it is a powerfull and versatile plasma code. I want to use it for my research.
Can you please answer my questions.

1. I am new to particle in cell simulation packages. I do not know how to use this kind of packages and I could not see any manuals that explain how to use vpic.. Can you please direct me to some kind of manuals etc. for learning how to use vpic for different applications?

2. Can I use vpic for dense plasma focus device simulation (especially pinch phase). I am mostly interested in pinch simulation. If it is possilbe, it will be great to simulate the whole plasma dynamics from beginning to until pinch phase.

3. I saw this paper (VPIC 2.0: Next Generation Particle-in-Cell Simulations). Can we have access to VPIC 2.0 code ?

Best regards
Thanks
Yasar Ay

The current way the stride is calculated for the checkpoint is broken in 2d (but works for 3d)

Hi,

I need to specify a non-uniform plasma density for my VPIC study. How to specify a linear density ramp in VPIC in x- direction ?

Please help me. Thanks.

Regards,

Ratan

Hi,

I want to use VPIC for my research work. I have installed it in my local linux machine. I am running using sample input deck. I see that data is printing and the output datas are being saved in different files lik ehydro hhydro field etc. Inside each folder there are several data T.0 T. 200 etc.

I want to know how to visualize these data and what type of data files are generating?

please help me.

Thank you.

Regards,
Ratan

instead there is a method called species which create a species_t

The regular particle dump writes some meta data using WRITE_HEADER_V0. The HDF5 dump should write similar meta data (to attributes in the HDF5 file?), including species ID, species name, charge and mass per particle, what timestep the output was written at, etc.

Right now the TA collisions accepts a users requests to run collisions on species with different weights. This runs but does not give a correct science answer as it is not supported by the current formulation of the TA operator

There is theory work by Miller and Combi that addresses this, but needs to be implemented

For now we can just detect the bad input and error out to avoid the impression that the code is doing the right thing when it's not

The Travis CI doesn't run on GitHub. Probably best to migrate to GitHub actions anyway.