sudo apt install curl
alternativelly can use wget in the Build_SDK.sh

Building EMsoft 5.0 with cmake version 3.12.1 I get

> cmake --version
cmake version 3.12.1
> cmake -DCMAKE_BUILD_TYPE=Release -DEMsoft_SDK=/home/hakonwii/emsoft/EMsoft_SDK -DEMsoft_ENABLE_EMsoftWorkbench=OFF /home/hakonwii/emsoft/EMsoft
-- The Current Build type being used is Release
-- BUILD_SHARED_LIBS: OFF
-- Support Lib Exists: /home/hakonwii/emsoft/EMsoft_SDK/jsonfortran-4.2.1-Release/lib64/cmake/jsonfortran-gnu-4.2.1
-- Support Lib Exists: /home/hakonwii/emsoft/EMsoft_SDK/bcls-0.1-Release/lib/cmake/bcls
-- Support Lib Exists: /home/hakonwii/emsoft/EMsoft_SDK/CLFortran-0.0.1-Release/lib/cmake/CLFortran
-- Support Lib Exists: /home/hakonwii/emsoft/EMsoft_SDK/hdf5-1.8.20-Release/share/cmake
-- ********* STARTING EMsoft CONFIGURATION ***********************
-- EMsoft_SDK Location: /home/hakonwii/emsoft/EMsoft_SDK
CMake Error at CMakeLists.txt:200 (FetchContent_MakeAvailable):
  Unknown CMake command "FetchContent_MakeAvailable".

The release notes for cmake version 3.14 (https://cmake.org/cmake/help/v3.14/release/3.14.html#modules) mentions this command as new in that version... Should cmake_minimum_required be adjusted to 3.14?

I just tried to do an EMEBSDDI run. The EBSD system we use is: OXFORD Instruments Nanoanalysis Symmetry and the software used to generate the files is Aztec.

When using the .h5oina files and setting the inputtype to OxfordHDF, the EMEBSDDI stops with the following error message:
----> Fatal error in routine openExpPatternFile: OxfordHDF input format not yet implemented

Do i have to implement the OxfordHDF format first? If yes how can I do this?

Or is there another possibility to run the indexing with the stored unprocessed pattern?

EMEBSDDI-Si.zip

What I mean by the title is that EMOpenClinfo returns an error for clGetDeviceInfo that I don't get when calling the same command from C++. This is a recent development, I had a kernel upgrade, but it doesn't explain the different behaviour. Yes, I did wipe down all nvidia and opencl drivers and reinstalled fresh and installed fresh sdk and EMsoft. Did not help. Anyway, here is some data:

$ EMOpenCLinfo 


 Program name         : EMOpenCLinfo.f90
 Purpose              : List OpenCL platform and device information
 Platform             : Linux
 Source code version  : 5_0_20200702_0
 Source code Revision : abfbdde
 Build Date/Time      : 2020-07-02 23:15:57Z

 See https://github.com/EMsoft-org/EMsoft/wiki for selected help pages.

 Jul  2 2020   7:25:28.510 PM


Number of Platforms: 1
--------
Platform: 1

 Profile: FULL_PROFILE
 Version: OpenCL 1.2 CUDA 10.2.185
 Name: NVIDIA CUDA
 Vendor: NVIDIA Corporation
 Extensions: cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics

  Non-fatal error: Error: CL_DEVICE_NOT_FOUND
 No CPU devices found on this platform


Num GPU Devices:  1


 ----> Fatal error in routine CLquery_platform_info:clGetDeviceInfo: Error: CL_INVALID_VALUE


STOP   Progam ended abnormally

$ clinfo

Number of platforms                               1
  Platform Name                                   NVIDIA CUDA
  Platform Vendor                                 NVIDIA Corporation
  Platform Version                                OpenCL 1.2 CUDA 10.2.185
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics
  Platform Extensions function suffix             NV

  Platform Name                                   NVIDIA CUDA
Number of devices                                 1
  Device Name                                     GeForce GTX 1650
  Device Vendor                                   NVIDIA Corporation
  Device Vendor ID                                0x10de
  Device Version                                  OpenCL 1.2 CUDA
  Driver Version                                  440.100
Device OpenCL C Version                         OpenCL C 1.2 
  Device Type                                     GPU
  Device Topology (NV)                            PCI-E, 01:00.0
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               16
  Max clock frequency                             1560MHz
  Compute Capability (NV)                         7.5
  Device Partition                                (core)
    Max number of sub-devices                     1
    Supported partition types                     None
    Supported affinity domains                    (n/a)
  Max work item dimensions                        3
  Max work item sizes                             1024x1024x64
  Max work group size                             1024
  Preferred work group size multiple              32
  Warp size (NV)                                  32
  Preferred / native vector sizes                 
    char                                                 1 / 1       
    short                                                1 / 1       
    int                                                  1 / 1       
    long                                                 1 / 1       
    half                                                 0 / 0        (n/a)
    float                                                1 / 1       
    double                                               1 / 1        (cl_khr_fp64)
  Half-precision Floating-point support           (n/a)
  Single-precision Floating-point support         (core)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  Yes
  Double-precision Floating-point support         (cl_khr_fp64)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
  Address bits                                    64, Little-Endian
  Global memory size                              4101898240 (3.82GiB)
  Error Correction support                        No
  Max memory allocation                           1025474560 (978MiB)
  Unified memory for Host and Device              No
  Integrated memory (NV)                          No
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       4096 bits (512 bytes)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        524288 (512KiB)
  Global Memory cache line size                   128 bytes
  Image support                                   Yes
    Max number of samplers per kernel             32
    Max size for 1D images from buffer            268435456 pixels
    Max 1D or 2D image array size                 2048 images
    Max 2D image size                             32768x32768 pixels
    Max 3D image size                             16384x16384x16384 pixels
    Max number of read image args                 256
    Max number of write image args                32
  Local memory type                               Local
  Local memory size                               49152 (48KiB)
  Registers per block (NV)                        65536
  Max number of constant args                     9
  Max constant buffer size                        65536 (64KiB)
  Max size of kernel argument                     4352 (4.25KiB)
  Queue properties                                
    Out-of-order execution                        Yes
    Profiling                                     Yes
  Prefer user sync for interop                    No
  Profiling timer resolution                      1000ns
  Execution capabilities                          
    Run OpenCL kernels                            Yes
    Run native kernels                            No
    Kernel execution timeout (NV)                 Yes
  Concurrent copy and kernel execution (NV)       Yes
    Number of async copy engines                  3
  printf() buffer size                            1048576 (1024KiB)
  Built-in kernels                                (n/a)
  Device Extensions                               cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics

NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  NVIDIA CUDA
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   Success [NV]
  clCreateContext(NULL, ...) [default]            Success [NV]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  No platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  No platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  Invalid device type for platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  No platform

ICD loader properties
  ICD loader Name                                 OpenCL ICD Loader
  ICD loader Vendor                               OCL Icd free software
  ICD loader Version                              2.2.11
  ICD loader Profile                              OpenCL 2.1

I can query device from C++:

 ----------------------------------
 Device GeForce GTX 1650
 ---------------------------------
  CL_DEVICE_NAME: 			GeForce GTX 1650
  CL_DEVICE_VENDOR: 			NVIDIA Corporation
  CL_DRIVER_VERSION: 			440.100
  CL_DEVICE_TYPE:			CL_DEVICE_TYPE_GPU
  CL_DEVICE_MAX_COMPUTE_UNITS:		16
  CL_DEVICE_MAX_WORK_ITEM_DIMENSIONS:	3
  CL_DEVICE_MAX_WORK_ITEM_SIZES:	1024 / 1024 / 64 
  CL_DEVICE_MAX_WORK_GROUP_SIZE:	1024
  CL_DEVICE_MAX_CLOCK_FREQUENCY:	1560 MHz
  CL_DEVICE_ADDRESS_BITS:		64
  CL_DEVICE_MAX_MEM_ALLOC_SIZE:		977 MByte
  CL_DEVICE_GLOBAL_MEM_SIZE:		3911 MByte
  CL_DEVICE_ERROR_CORRECTION_SUPPORT:	no
  CL_DEVICE_LOCAL_MEM_TYPE:		local
  CL_DEVICE_LOCAL_MEM_SIZE:		48 KByte
  CL_DEVICE_MAX_CONSTANT_BUFFER_SIZE:	64 KByte
  CL_DEVICE_QUEUE_PROPERTIES:		CL_QUEUE_OUT_OF_ORDER_EXEC_MODE_ENABLE
  CL_DEVICE_QUEUE_PROPERTIES:		CL_QUEUE_PROFILING_ENABLE
  CL_DEVICE_IMAGE_SUPPORT:		1
  CL_DEVICE_MAX_READ_IMAGE_ARGS:	256
  CL_DEVICE_MAX_WRITE_IMAGE_ARGS:	32

and with pyopencl:

============================================================
OpenCL Platforms and Devices
============================================================
Platform - Name:  NVIDIA CUDA
Platform - Vendor:  NVIDIA Corporation
Platform - Version:  OpenCL 1.2 CUDA 10.2.185
Platform - Profile:  FULL_PROFILE
    --------------------------------------------------------
    Device - Name:  GeForce GTX 1650
    Device - Type:  ALL | GPU
    Device - Max Clock Speed:  1560 Mhz
    Device - Compute Units:  16
    Device - Local Memory:  48 KB
    Device - Constant Memory:  64 KB
    Device - Global Memory: 4 GB
    Device - Max Buffer/Image Size: 978 MB
    Device - Max Work Group Size: 1024

I'm out of ideas. Is clfortran looking somewhere else for drivers?

The SDK compiled and built correctly with no errors.
Configuring the EMsoft package gives the following errors

This was easy to fix by setting it to the same directory as CLFORTRAN_DIR

The next issue was with Eigen with the Eigen3_DIR CMake variable not found. But the Eigen build did not have a cmake configuration file inside the SDK. Has anyone else grappled with this problem?

While running the EMEBSDDI program the experimental pattern are processed but the ADP is not calculated.

It works with the provided test files, but not with own data sets.

Since we have an Oxford EBSD system we can only use .ebsp files as pattern format (OxfordBinary). Could this be the reason why no output files are calculated/ created?
EMEBSDDI-run_Si.zip
EMEBSDDI-Si.nml.zip

Communicated b Tijmen Vermeji

needs to have an additional line:

NumberFamilies 0

immediately after writing the lattice parameters

Tried installing pyEMsoft from the latest commit 6606859, but failed. Can some get pyEMsoft running on Ubuntu? If so, what does your Python env (Anaconda/venv) look like?

I've followed the installation instructions (https://pyemsoftreadthedocs.readthedocs.io/en/latest/Installation.html).

From an amateur's glance, I believe I've found some minor things in Source/pyEMsoft/run_pyEMsoft.sh.in and kind_map which should be changed:

1. Explanation before code (line 47) should read: "THIS FILE IS AUTOMATICALLY... LOCATED AT @EMsoft_SOURCE_DIR@/Source/pyEMsoft/run_pyEMsoft.sh.in", adding the Source
2. CLsupport.f90 is not located together with the other source files in EMsoftLib, but in its own EMOpenCLLib directory? Therefore it must either be added both places, or the bash script must be updated.
3. Locations of local.f90 and stringconstants.f90 is not EMsoftBuild/Debug/EMsoft/EMsoftLib, but EMsoftBuild/Debug/EMsoftLib. Line 106 in run_pyEMsoft.sh.in should be updated.
4. The kind_map misses the character entries '34':'char' and '38':'char'? I've gotten these lines at the bottom of build_error.log:

RuntimeError: Unknown combination of type "character" and kind "34" - add to kind map and try again
f90wrap: RuntimeError('Unknown combination of type "character" and kind "34" - add to kind map and try again')
         for help use --help

and, after adding 34, I get the same error with 38.

After addressing this and the source file locations, the script runs for ~1-2 hours without any errors being printed to stdout. However, Python complains saying that _pyEMsoft is not found. Looking in build_error.log, I find these lines at the bottom:

analyzevars: charselector={'len': '1'} unhandled.analyzevars: charselector={'len': '1'} unhandled.analyzevars: charselector={'len': '1024'} unhandled.analyzevars: charselector={'len': '1024'} unhandled.analyzevars: charselector={'len': '1024'} unhandled.analyzevars: charselector={'len': '1'} unhandled.analyzevars: charselector={'len': '1'} unhandled.analyzevars: charselector={'len': '1'} unhandled.analyzevars: charselector={'len': '1'} unhandled.analyzevars: charselector={'len': '1024'} unhandled.analyzevars: charselector={'len': '1024'} unhandled.analyzevars: charselector={'len': '1024'} unhandled.analyzevars: charselector={'len': '1'} unhandled.analyzevars: charselector={'len': '1'} unhandled.analyzevars: charselector={'len': '6'} unhandled.append_needs: unknown need 'char'
error: Command "gcc -pthread -B /home/hakon/miniconda3/envs/kp-dev/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DSCIPY_MKL_H -DHAVE_CBLAS -I/home/hakon/kode/emsoft/EMsoftBuild/Debug/EMsoftLib -I/home/hakon/kode/emsoft/EMsoftBuild/Debug/EMsoft/EMsoftHDFLib -I/home/hakon/kode/emsoft/EMsoft_SDK/hdf5-1.10.5-Debug/include/static -I/home/hakon/kode/emsoft/EMsoft_SDK/CLFortran-0.0.1-Debug/include -I/home/hakon/kode/emsoft/EMsoft_SDK/jsonfortran-4.2.1-Debug/include -I/home/hakon/kode/emsoft/EMsoft_SDK/fftw-3.3.8/include -I/home/hakon/miniconda3/envs/kp-dev/include -I/tmp/tmpic0h8u7u/src.linux-x86_64-3.8 -I/home/hakon/miniconda3/envs/kp-dev/lib/python3.8/site-packages/numpy/core/include -I/home/hakon/miniconda3/envs/kp-dev/include/python3.8 -c /tmp/tmpic0h8u7u/src.linux-x86_64-3.8/_pyEMsoftmodule.c -o /tmp/tmpic0h8u7u/tmp/tmpic0h8u7u/src.linux-x86_64-3.8/_pyEMsoftmodule.o -MMD -MF /tmp/tmpic0h8u7u/tmp/tmpic0h8u7u/src.linux-x86_64-3.8/_pyEMsoftmodule.o.d" failed with exit status 1

I guess they are related to what's explained in the installation guide: https://pyemsoftreadthedocs.readthedocs.io/en/latest/Installation.html#debugging, but I have too little knowledge to tell.

I've tried reinstalling the newest version/installing older versions and all have given me this error message.

Machines with more than 1 platform are not able to select a device to run MonteCarlo simulations.

Two issues I noticed, both for the case of eu = (pi, pi, pi/2)

1. the single and double version of eu2ax gives different signs of the axis
   single version returns (-1/sqrt(2), -1/sqrt(2), 0, pi)
   double version returns (1/sqrt(2), 1/sqrt(2), 0, pi)

2. for these Euler angles, successive operations of eu2ax followed by ax2eu returns (pi/2, pi, 0). If these Euler angles are used to generate the rotation matrix, then it agrees with the rotation matrix corresponding to the other euler angle, which is the way its implemented in MODRotationsTest. However, the axis-angle pair (and therefore quaternions, Rodrigues vector etc. do not agree, and differ in sign of the rotations axis).

All of the came up in my implementation of the exponential map orientation representation and implementing the pair wise and triple tests.

The Oxford ebsp reader in the patternmod module functions correctly when reading a row of patterns, but when reading a single pattern, there is a clear vertical offset that depends on where the pattern is located in the field of view.

I just started using the EMsoft 5_0_20200507.

After doing the configurations, i wanted to start EMMCOpenCL via

EMMCOpenCL -t

But unfortunately it's not working and the following error message shows up:

Fatal error in routine CopyTemplateFiles: template code file not found:D:\EMsoft-5.0.20200507.-Win64\resources\templatecodes.txt

Since I just started working with console commands and EMsoft i don`t really know why it is not working.

I`m starting the program from the correct path (D:\EMsoft-5.0.20200507.-Win64\bin>)
and there is also a folder with the templatecodes included

Many of the EBSD pattern using EDAX are collected with optimization -" Reduced noise" which uses a 5 by 5 binning on EBSD camera. However, the EMsoft currently doest not support indexing such patterns. It will be highly appreciated if EMsoft can index 5 by 5 binned patterns. #

On machine using Intel Parallel Studio XE 2018 Update 3 Composer Edition for Fortran Windows. I've had to make a few changes to get things working

I've added the following two explicit array allocations

 allocate(resultarray(Nd))
 allocate(indexarray(Nd))

And then modified the deallocations to the following

 deallocate(ppend, ppendE, dict, res, results1, results2) 
 deallocate(resulttmp, expt, dicttranspose, resultarray)
 deallocate(indexarray)
 nullify(results)
 nullify(dpsort)
 nullify(indexlist)
 nullify(dpindex)

I also modified the corresponding include file so that the ipar and indexmain are both int32_t pointers as opposed to size_t pointers

Hello!

Running EMEBSDDI on Windows with binaries from Bluequartz (EMsoft-5.0.20200714.-Win64.zip) works nicely, but on Linux with the same pattern and NML file fails with the output:

> EMEBSDDI sic_4h_ebsddi.nml 


 Copyright (C) 2001-2019 Marc De Graef Research Group/CMU
 EMsoft comes with ABSOLUTELY NO WARRANTY.
 This is free software, and you are welcome to redistribute it
 under certain conditions; see License.txt file for details.


 Program name         : EMEBSDDI.f90
 Purpose              : Program to index EBSD patterns using a dynamically calculated pattern dictionary
 Platform             : Linux
 Source code version  : 5_0_20200714_
 Source code Revision : 
 Build Date/Time      : 2020-07-14 15:49:24Z

 See https://github.com/EMsoft-org/EMsoft/wiki for selected help pages.

 Jul 14 2020   6:56:34.695 PM


Number of command line arguments detected:           1
 Array size analysis
 ===================
 Ne           :         1024
 Nd           :         1024
 L            :         9216
 size result array  :      4194304
 size_in_bytes_dict :              37748736
 size_in_bytes_expt :              37748736
 Total allocations on GPU (Mb):                    76
 reading from xtalfile si_carbide_4h\si_carbide_4h.xtal
Orientation space sampling mode set to RFZ
Point group number and number of cubochoric sampling points :  25,  100
 Number of unique orientations sampled =        :  666187
--> Initializing OpenCL device
--> Allocating various arrays for indexing
Preprocessing experimental patterns
Creating temporary file :/home/hakon/.config/EMsoft/tmp/1_4h.data
 -> Number of threads set to  4
Starting processing of experimental patterns
 pattern size :  96 x   96
Completed row   5 of  350 rows
[...]
Completed row 350 of  350 rows
 -> experimental patterns preprocessed
Number of experimental patterns processed per second :    4819.7

 -> computing Average Dot Product map (ADP)

 -> Number of threads set to  4
 Jul 14 2020   6:57:43.601 PM
  actual number of OpenMP threads  =            4
Indexing duration (system_clock, s)                :
         3.514
Number of pattern comparisons per second           :
**************
Number of experimental patterns indexed per second :
     20916.336

At line 899 of file /home/hakon/kode/emsoft/EMsoft/Source/EMsoftHDFLib/EMh5ebsd.f90
Fortran runtime error: Index '0' of dimension 2 of array 'eulerarray' outside of expected range (1:666624)

Error termination. Backtrace:
#0  0x7f8fd444ecd1 in ???
#1  0x7f8fd444f819 in ???
#2  0x7f8fd444fe96 in ???
#3  0x5599c41a5320 in __emh5ebsd_MOD_h5ebsd_writefile
	at /home/hakon/kode/emsoft/EMsoft/Source/EMsoftHDFLib/EMh5ebsd.f90:899
#4  0x5599c411e868 in EBSDDIdriver
	at /home/hakon/kode/emsoft/EMsoft/Source/EMOpenCLLib/Indexingmod.f90:1288
#5  0x5599c40d22b9 in emebsddi
	at /home/hakon/kode/emsoft/EMsoft/Source/DictionaryIndexing/EMEBSDDI.f90:86
#6  0x5599c40d22f6 in main
	at /home/hakon/kode/emsoft/EMsoft/Source/DictionaryIndexing/EMEBSDDI.f90:54

I would have though I did something wrong, however it works fine on Windows... Anyone experiencing the same on Linux?

should be useful for EBSD, ECP, and TKD dictionary indexing output

Hi,

I have found an issue in EMTKD - incorrect rotations of the beam intensity.

At first, I was not sure what is the correct convention for the sample tilt - I would use -20°, Wiki tutorial on TKD says +20°. Anyway, I produced both masterfiles for alpha-Ti with EMMCfoil.txt, changing only the sig to +/-20°, processed through EMTKDmaster and plotted using EMTKD with EMTKD.txt (for simplicity - pattern centered on detector) as a series with detector tilt from -70° to 110°. The (0001) basal plane of Ti is parallel with the foil, Euler angles (0.0, 0.0, 0.0). Orientation of the patterns is the same as when viewed by HDFview.

Now, the Kikuchi pattern seems to be rotated correctly, while the dark band, which I assume is an intensity singularity in the foil plane, is moving in an opposite direction. (It could actually be the opposite way, if I got the pattern orientation upside-down - intensity would be rotated correctly and master pattern not.)

Source code version : 5_0_20201010_ (Win64 build)

EMFitOrientationPS produces Segmentation fault when keeping pre-processed patterns in memory ( inRAM = .TRUE. )

Same problem occurs in v4.0 and v4.1 Public releases.

It would be nice to have the option to generate a montage image of individual ECCI images directly from the EMECCI program instead of having to do this with an external program that reads the output HDF5 file.

It would be nice to have a C++ wrapper routine for the EBSD pattern refinement program...

-- EMsoft_SDK Location: /Software/EMsoft_SDK
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- The Fortran compiler identification is GNU 9.3.0
...
-- Check size of long long
-- Check size of __int64
-- Check size of __int64 - failed
...
-- Check size of off64_t
-- Check size of off64_t - failed
...
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
CMake Error at /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
Could NOT find BLAS (missing: BLAS_LIBRARIES)

I experience worse EBSDDI indexing results with EMsoft 5.0 than with 4.2, i.e. a lower Indexing Success Rate (ISR) of 64% compared to 100% for the latter, with the same Ni experimental patterns, master pattern and NML input file.

After going through the nightly builds for Windows and re-indexing the same ROI, the ISR dropped to 64% with the nightly build 2019-09-13. For 2019-09-12 the ISR was 100%.

Does anyone else experience this? I don't know the root of the problem. I see that a series of commits with changes to the point of view (EBSD-detector) were made on the 12th. Do I have to change my pattern centre definition after these changes?

Relevant files are available here: http://folk.ntnu.no/hakonwii/files/emsoft_issue/ (dot product and ang files produced by the nightly builds are in the directories 4_3_9_12 and _13).

The current implementation of the Poisson random number generator sometime has issues and is generally rather slow; should be replaced by a more efficient routine, for instance the code from https://jblevins.org/mirror/amiller/random.f90

I used your nightly builds to do some EBSD simulations. I started with your example from the wiki.
Using the actual 5.0.2 version I get in the monte carlo simulation the error:
Kernel source length (characters) : 13566
----> Fatal error in routine DoMCsimulation:clCreateProgramWithSource: Error: CL_INVALID_CONTEXT
Progam ended abnormally
EMMCOpenCL Completed
The 5.0.0 version worked fine.
So I tried EMOpenCLinfo.exe in the different nightly builds:
Up to version 5_0_20191102_0 all is fine. (Output on top of file bug.txt.)
From version 5_0_20191106_0 on I get a endless loop of wrong output.
See bug.txt
bug.txt

I'm still on Ubuntu 19.04 and some of the Debug checks on Public are failing (Private one is not building at all for me, so I'm trying to take it step by step). One of them is the fact that it fails to find __int64 which is a Windows type I think.

-- Check size of __int64
-- Check size of __int64 - failed

Here are my compilers:

$ dpkg --list | grep compiler
ii  g++                                              4:8.3.0-1ubuntu3                                     amd64        GNU C++ compiler
ii  g++-8                                            8.3.0-6ubuntu1                                       amd64        GNU C++ compiler
ii  gcc                                              4:8.3.0-1ubuntu3                                     amd64        GNU C compiler
ii  gcc-8                                            8.3.0-6ubuntu1                                       amd64        GNU C compiler
ii  gfortran                                         4:8.3.0-1ubuntu3                                     amd64        GNU Fortran 95 compiler
ii  gfortran-8                                       8.3.0-6ubuntu1                                       amd64        GNU Fortran compiler

I'll try to take a look but I don't really know what I'm doing. Any suggestions would be much appreciated.

I want to perform CCEBSD with the simulated patterns via openXY. But the simulated patterns are very different from the experimental patterns. Are these differences caused by deformation tensor?

Actually, I don't know the exact CCD pixel size. I use a default value 60.

The pattern center optimized by Aztec is : Pcx = 0.48, Pcy=0.548, z = 0.508
The ctf and cpr files have also been included in the file named 'attachment.zip'
attachment.zip

Experimental:

Simulation:

&EBSDdata
! template file for the EMEBSD program
!
! distance between scintillator and illumination point [microns]
L = 15384.96,
! tilt angle of the camera (positive below horizontal, [degrees])
thetac = 10.0,
! CCD pixel size on the scintillator surface [microns]
delta = 60.0,
! number of CCD pixels along x and y
numsx = 622,
numsy = 512,
! pattern center coordinates in units of pixels
xpc = -12.44,
ypc = 24.576,
! angle between normal of sample and detector
omega = 0.0,
! transfer lens barrel distortion parameter
alphaBD = 0.0,
! energy range in the intensity summation [keV]
energymin = 10.0,
energymax = 20.0,
! include a realistic intensity background or not ...
includebackground = 'n',
! name of angle file (euler angles or quaternions); path relative to EMdatapathname
anglefile = 'euler.txt',
! does this file have only orientations ('orientations') or does it also have pattern center and deformation tensor ('orpcdef')
! if anglefiletype = 'orpcdef' then each line in the euler input file should look like this: (i.e., 15 floats)
! 55.551210 58.856774 325.551210 0.0 0.0 15000.0 1.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 1.00
! <- Euler angles (degrees) -> <- pat. ctr. -> <- deformation tensor in column-major form->
!
anglefiletype = 'orientations',
! 'tsl' or 'hkl' Euler angle convention parameter
eulerconvention = 'hkl',
! name of EBSD master output file; path relative to EMdatapathname
masterfile = 'Al_master_20kV.h5',
energyfile = 'Al_master_20kV.h5',
! name of output file; path relative to EMdatapathname
datafile = '11-04-2.h5',
! bitdepth '8bit' for [0..255] bytes; 'float' for 32-bit reals; '##int' for 32-bit integers with ##-bit dynamic range
! e.g., '9int' will get you 32-bit integers with intensities scaled to the range [ 0 .. 2^(9)-1 ];
! '17int' results in the intensity range [ 0 .. 2^(17)-1 ]
bitdepth = '8bit',
! incident beam current [nA]
beamcurrent = 150.0,
! beam dwell time [micro s]
dwelltime = 100.0,
! include Poisson noise ? (y/n) (noise will be applied before binning and intensity scaling)
poisson = 'n',
! binning mode (1, 2, 4, or 8)
binning = 1,
! should we perform an approximate computation that includes a lattice distortion? ('y' or 'n')
! This uses a polar decomposition of the deformation tensor Fmatrix which results in
! an approcimation of the pattern for the distorted lattice; the bands will be very close
! to the correct position in each pattern, but the band widths will likely be incorrect.
applyDeformation = 'n'
! if applyDeformation='y' then enter the 3x3 deformation tensor in column-major form
! the default is the identity tensor, i.e., no deformation
Ftensor = 1.D0, 0.D0, 0.D0, 0.D0, 1.D0, 0.D0, 0.D0, 0.D0, 1.D0,
! intensity scaling mode 'not' = no scaling, 'lin' = linear, 'gam' = gamma correction
scalingmode = 'gam',
! gamma correction factor
gammavalue = 2.5,
!=======================
! if the 'makedictionary' parameter is 'n', then we have the normal execution of the program
! if set to 'y', then all patterns are pre-processed using the other parameters below, so that
! the resulting dictionary can be used for static indexing in the EMEBSDDI program...
! these parameters must be taken identical to the ones in the EMEBSDDI.nml input file to have
! optimal indexing...
makedictionary = 'n',
! should a circular mask be applied to the data? 'y', 'n'
maskpattern = 'n',
! mask radius (in pixels, AFTER application of the binning operation)
maskradius = 240,
! hi pass filter w parameter; 0.05 is a reasonable value
hipassw = 0.05,
! number of regions for adaptive histogram equalization
nregions = 10,
!=======================
! number of threads (default = 1)
nthreads = 10,
/

EMECPsingle tries to write a six-character string to an output HDF file; that particular variable is no longer necessary so that bit of code should probably be removed.

Clicking the "+" button does not work correctly. The user needs to use the line command program to build up the xtal file.

The maximum dot product listed on the command line during the program runs does not appear to agree with the actual largest dot products in the output file. Perhaps there is an issue with the sorting code?

tested only on Pt; below 4 kV, the master pattern starts to have negative intensities which is not physical. Not sure what causes this.

Using EMEBSDDIpreview with pattern data at the very top level will result in read warnings on GFortran and a crash on IFort compiled versions of EMsoft.

From the GFortran we get:

 Averaging patterns over  1 by   1 area
Error code :          -1
   returned by routine HDF_readHyperslabIntegerArray3D:h5dopen_f:                ```

Same warnings with IFort but ends in a crash.

Windows uses the 'red oldname newname' command to rename a file, not the UNIX 'mv' command. This causes a problem when trying to create a new configuration file if an old file is already present (Windows platform only).

Hi! I'm trying to compile on Ubuntu 18.10 after a successful SDK build. Following the instructions in EMsoft/ReadMe.md, building with make -j4 fails in EMsoftHDFLib:

Scanning dependencies of target EMsoftHDFLib
[ 27%] Building Fortran object Utilities/CMakeFiles/EMlistSG.dir/EMlistSG.f90.o
[ 27%] Building Fortran object Utilities/CMakeFiles/EMOpenCLinfo.dir/EMOpenCLinfo.f90.o
[ 27%] Building Fortran object Utilities/CMakeFiles/EMsoftinit.dir/EMsoftinit.f90.o
make[2]: Circular EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/EMh5ebsd.f90.o <- EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/ebsddimod.mod.stamp dependency dropped.
[ 27%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/HDFsupport.f90.o
[ 28%] Linking Fortran executable ../Bin/EMsoftinit
[ 28%] Built target EMsoftinit
[ 29%] Linking Fortran executable ../Bin/EMOpenCLinfo
[ 29%] Linking Fortran executable ../Bin/EMlistSG
[ 29%] Built target EMOpenCLinfo
[ 29%] Built target EMlistSG
[ 29%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/NameListHDFwriters.f90.o
[ 29%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/utilities.f90.o
[ 29%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/InitializersQCHDF.f90.o
[ 29%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/initializersHDF.f90.o
[ 29%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/TKDDImod.f90.o
[ 29%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/EBSDiomod.f90.o
[ 30%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/EBSDmod.f90.o
[ 30%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/ECPmod.f90.o
[ 30%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/TKDmod.f90.o
[ 31%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/ECPiomod.f90.o
[ 32%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/PFInversionHDF.f90.o
[ 32%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/EMh5ebsd.f90.o
[ 32%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/FitOrientations.f90.o
[ 33%] Building Fortran object EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/QCmodHDF.f90.o
/home/hakon/kode/emsoft/EMsoft/Source/EMsoftHDFLib/EMh5ebsd.f90:786:4:

 use ebsddimod
    1
Fatal Error: Can't open module file ‘ebsddimod.mod’ for reading at (1): Fila eller mappa finnes ikke
compilation terminated.
make[2]: *** [EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/build.make:167: EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/EMh5ebsd.f90.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/Makefile2:1122: EMsoftHDFLib/CMakeFiles/EMsoftHDFLib.dir/all] Error 2
make: *** [Makefile:163: all] Error 2

Seems like the circular dependency was added in commit 67125a6a73d5e8fbd67b50c0bff08f97e7e5afae six days ago. After reverting to the commit before this one, compilation was successful with just two warnings.

I'm not familiar with Fortran, how to solve this?

I can read the .ang file produced by angebsd_writeFile in EBSDiomod.f90 in OIM Analysis 7 only after I've change the order of the pattern position columns x, y in the .ang file, by changing the order of when they are written to file.

Before:

[...]
! sampling coordinates
    if (sum(ebsdnl%ROI).ne.0) then
      write(str4,'(A,F12.5)') ' ',float(floor(float(ii-1)/float(ebsdnl%ROI(3))))*ebsdnl%StepY
      write(str5,'(A,F12.5)') ' ',float(MODULO(ii-1,ebsdnl%ROI(3)))*ebsdnl%StepX
else
[...]

After:

[...]
! sampling coordinates
    if (sum(ebsdnl%ROI).ne.0) then
      write(str4,'(A,F12.5)') ' ',float(MODULO(ii-1,ebsdnl%ROI(3)))*ebsdnl%StepX
      write(str5,'(A,F12.5)') ' ',float(floor(float(ii-1)/float(ebsdnl%ROI(3))))*ebsdnl%StepY
    else
[...]

Have EDAX changed the order of the x, y columns after version 7 of OIM Analysis?

Firstly, the start fit function in Workbench by BlueQuartz is not functioning. So I have to use EMDPFit.

In the EMSPFit.nml file, some parameters are undefined. This is found when running EMDPFit, an error is displayed, for example, Cannot match nameless object name energyfile. When I looked into the NameListHandlers.f90, there are subroutines GetEMDPFit4NameList and GetEMDPFitNameList. And in the latter (GetEMDPFitNameList), energyfile is indeed not defined as a parameter. After deleting this option in the .nml file and some other ambiguous parameters. EMDPFit can run. The parameters that need to be changed are:

• change exptfile_pat1 to exptfile, and delete exptfile_pat2 (and 3, 4).
• change phi1_pat1 to phi1, the same goes for phi and phi2, and delete the rest with _pat2 ( and 3, 4).
• change stepx and stepy to step_xpc and step_ypc.

Although EMDPFit runs after changing the .nml file, it seems not fitting. The display of "Least value of F" is always NaN. I think probably bobyqa.f90 needs to be modified, since F is computed there.

Either Workbench or EMDPFit needs to be fixed so we can fit the pattern center.

Thanks.

It seems that there's functionality for computing a histogram of CSL or random grain boundary pair angles (EMBGO) and for computing a pairwise distance matrix (EMGBOpd). It would be great if EMGBO or EMGBOpd (preferably the first) could be modified to take a user-specified list of grain boundary pairs, similar to the MATLAB implementation GBdist.m. Is this functionality present, but I'm just missing it? (I assume EMoSLERP doesn't have it either, which I had trouble finding documentation for).

Alternatively, I'm considering plugging single pair text files into EMGBOpd; however, I'm worried about whether or not that will result in a significant slow-down.

Is there some documentation that I'm missing? It seems like I may need an input file, but I'm having trouble finding the format of said file.
https://github.com/EMsoft-org/EMsoft/blob/develop/Source/GBs/EMoSLERP.f90

In the context of indexing with overlap master patterns, it would be nice to have the option in the so3.f90 module to change the orientation of the fundamental zone to an arbitrary orientation before applying the sampling algorithm. This will allow for sampling in cases where there is an orientation relation (e.g., the Burgers relation in Ti) for which the effective FZ does not fall in the standard orientation (e.g., for the Burgers OR, the monoclinic FZ is rotated by 45° around the cubic [001] axis). This would allow for sampling of an arbitrary FZ in the dictionary indexing approach, and should just take one additional (optional) argument in the sampling routines.

Hi all,

preparing to get people to use EMsoftWorkbench in our lab. However, EMsoftWorkbench nightly build 2020-06-16 (today) doesn't allow me to enter period as decimal sign, only comma, as a decimal sign. But running any program via the "Generate" button with a comma in a field throws an error (issue), forrtl: severe (157): Program Exception - access violation.

Any way around this that anyone knows of?

Hello,

I am having issues with the initial configuration of the EMsoft packages. When I run the EMsoftini.exe program I get an error message after inserting the user name, see attached pictures. When I look inside the EMsoftConfig.json file I see that not all the fields are automatically generated. Also, there is no back-facing curly bracket at the end of the document, suggesting me that something went wrong.

Even if the configuration is clearly not completed, I am still able to run a limited number of programs, but not all of them.
Is this issue known or am I doing something wrong? Thank you in advance for your help.

Regards,

Stefano

Stuart Wright reported memory leaks when calling the init_HIPassFilter and applyHIpassFilter routines; likely these are due to the use of fftw calls.

Hi, I was wondering if it's still possible to do Bloch wave simulations of CBED patterns using EMSoft (it looks like the answer is yes from the source code) and if so - is there a usage example anywhere? Many thanks!

The EMECP program produces patterns with a clear off-by-one error when writing byte-scaled patterns to the HDF file; error does not occur when writing floating point arrays.