Add --developer configure option to setup build and environment suitable for development.

Directory layout

Do not run make install for PETSc

* dependencies
    * include
    * lib
    * bin
* petsc
* pylith (plus pyre, nemesis, spatialdata)
    * include
    * lib
    * bin

• Make sure all source files are present, i.e., documentation.

macOS no longer includes Python v2 so we need to add Python to the PyLith binary.

Add note about VS Code integration that you must open the folder of the top-lvel source directory of a repository. Verify header files, etc are found.

Create docker images with all dependencies (except pyre, nemesis, spatialdata, PETSc, and PyLith).

• pylith-testenv-debian-stable-gcc8 (mpich)
• pylith-testenv-debian-testing-gcc9(openmpi)
• pylith-testenv-debian-testing-clang8(installer: openmpi)
• pylith-testenv-debian-testing-clang9 (installer: mpich)
• pylith-testenv-centos-7(mpich)
• pylith-testenv-centos-8(mpich)
• pylith-testenv-ubuntu-18.04 (openmpi)
• pylith-testenv-ubuntu-19.10(mpich)
• pylith-testenv-fedora-30 (openmpi)
• pylith-testenv-fedora-31 (mpich)

Description

We use distutils.sysconfig.parse_makefile() which was removed in Python v3.12 in packager/build.py.

Proposed solution

Write a simpleparse_makefile_vars() function for this functionality. We just need PACKAGE, VERSION, and abs_top_srcdir variables.

The MPI compiler wrappers embed the include path to the MPI install location. If a user has done a make install and then tries to run make after editing a header file, the build will see the installed header file, not the updated one.

The short-term workaround is to run make install in the library subdirectory to install the header files, so that there is no difference between the installed header files and the edited ones.

We should separate the dependencies (including MPI) from the packages that are likely to be edited or need updating by the user (spatialdata, PETSc, PyLith).

Installing the netCDF4 Python module via pip fails with because the numpy Python module is not found when the setup.py script tries to import it.

The Python sys.path is not getting the correct path inside the setup.py script. It has the Python directory, but not the site-packages directory under the Python directory. The workaround is to install via python3 setup.py install rather than using pip.

The MPI wrappers cause problems when trying to develop code ina build created by the installer; see #9.

The solution is to install all of the dependencies that won't be changed to one install directory and spatialdata, PETSc, and PyLith (which won't be changed) to another install directory.

• Add --enable-debugging configure options. Trigger debugging flags for configure of spatialdata, PETSc, and PyLith.

Add --debug configure option to build with debugging support in PETSc, spatialdata, and PyLith. Keep other dependencies optimized.

Attempting to run ParaView installed using the paraview debian package results in errors associated with OpenGL libraries.

Potential options that need researching:

• Installing (and configuring?) appropriate libraries in Docker container for software rendering.

• Using x11docker to run a second X server. I believe the user must install this on the host, so this approach may be more work for the user.

• Replace absolute path in first line of scripts in bin with /usr/bin/env
• Add libffi to binary build (Python dependency)

• Update base image to Ubuntu 22.04
• Update m4 URLS now that aren't mirroring on gitlab
• Create a development container supporting both Linux/amd64 and Darwin/arm64

See Docker documentation

Install dependencies

Use Ubuntu 20.04 w/USER=pylith-dev

Additional packages:

• gnupg2
• uncrustify
• autopep8
• matplotlib

Install any dependencies built form source into /opt/dependencies

CIG code

Create Docker volume for CIG code /opt/pylith

/opt
└── pylith
    ├── src
    │    ├── pythia
    │    ├── spatialdata
    │    ├── petsc
    │    └── pylith
    ├── build
    │   ├── debug
    │   │   ├── pythia
    │   │   ├── spatialdata
    │   │   └── pylith
    │   └── opt
    │       ├── pythia
    │       ├── spatialdata
    │       └── pylith
    └── dest
        ├── debug
        │   ├── bin
        │   ├── include
        │   ├── lib
        │   └── share
        └── opt
            ├── bin
            ├── include
            ├── lib
            └── share

Use --filter=blob:none when cloning.

Proj 6.3
CppProj 1.15.1
NetCDF 4.7.3
HDF5 1.10.6
MPICH 3.3.2
cmake 3.16.3

libssl and libcrypto for the Python hashtable module are not being built. We have copied them by hand from the build slave into the PyLith distribution. We should build these from source.

We need to check for any other dependencies that may be included indirectly (Python modules).

Description

The current description of the Git workflow does not clearly separate setup, building, and updating.

Proposed solution

Reorganize the description of the Git workflow into clearly distinguishable phases: setup, building, and updating.

Add diagram to manual with detailed list of dependencies (based on installer).

Python depends on libcrypt and there are now multiple versions floating around, so we can't rely on the system library. Fedora seems to be the first Linux distribution to move to libcrypt.so.2.

See pypa/manylinux#305 for some detailed discussion of this issue and compatibility.

Create a Python script to build/manage developer source.

• Use Python script to manage git repositories and rebuilding for spatialdata, PETSc, and PyLith
  • Update repos
  • (Re)configure
  • (Re)build
  • Either run commands or show commands to run.
  • Check that environment is setup
    • run nemesis
    • import pylith stuff
  • Debugging and optimized options

developer_helper Python script

--clone=[pythia, spatialdata, petsc, pylith, all]
--configure=[pythia, spatialdata, petsc, pylith, all]
--build=[pythia, spatialdata, petsc, pylith, all]
--pull=[pyre, nemesis, spatialdata, petsc, pylith, all]
--with-optimized [for configure, always include debugging symbols]
--show-config
--merge-from-upstream=[pythia, spatialdata, pylith]
--set-upstream=[pythia, spatialdata, petsc, pylith, all]

Show command or show and run command.

## Build functionality
* Clone
* Configure
* Build
    * debug / optimized

## Git functionality
* Set upstream
* Merge from upstream (git pull upstream BRANCH merges to current branch)
* Clone
* Pull

## Config file

[base]
src_dir = /opt/pylith/src
build_dir = /opt/pylith/build
install_dir = /opt/pylith/dest

[petsc]
protocol = ssh/https
repo = https://...
branch = knepley/pylith
upstream = False

[pythia]
protocol = ssh/https
repo_url = https://
branch = main
upstream = False

[spatialdata]
protocol = ssh/https
repo_url = https://...
branch = main
upstream = False

[pylith]
protocol = ssh/https
repo_url = https://...
branch = knepley/feature-petsc-fe
upstream = https://github.com/geodynamics/pylith.git

Comment from Josimar on cig-short:

I am trying to build pylith-2.2.1 from source but the compilation stops at the h5py installation. My machine runs on macOS v. 10.13.6. Note that I was able to build it on a linux computer without any problems.

A summary of the error is below and the complete full set log is attached. Would you have any suggestions to fix this issue ?

Error summary:

Download error on https://pypi.org/simple/pkgconfig/: unknown url type: https -- Some packages may not be found!
Couldn't find index page for 'pkgconfig' (maybe misspelled?)
Download error on https://pypi.org/simple/: unknown url type: https -- Some packages may not be found!
No local packages or working download links found for pkgconfig
Traceback (most recent call last):
  File "setup.py", line 167, in <module>
    cmdclass = CMDCLASS,
  File "build/bdist.macosx-10.13-x86_64/egg/setuptools/__init__.py", line 128, in setup
  File "build/bdist.macosx-10.13-x86_64/egg/setuptools/__init__.py", line 123, in _install_setup_requires
  File "build/bdist.macosx-10.13-x86_64/egg/setuptools/dist.py", line 504, in fetch_build_eggs
  File "build/bdist.macosx-10.13-x86_64/egg/pkg_resources/__init__.py", line 774, in resolve
  File "build/bdist.macosx-10.13-x86_64/egg/pkg_resources/__init__.py", line 1057, in best_match
  File "build/bdist.macosx-10.13-x86_64/egg/pkg_resources/__init__.py", line 1069, in obtain
  File "build/bdist.macosx-10.13-x86_64/egg/setuptools/dist.py", line 571, in fetch_build_egg
  File "build/bdist.macosx-10.13-x86_64/egg/setuptools/command/easy_install.py", line 667, in easy_install
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pkgconfig')
make[2]: *** [h5py] Error 1
make[1]: *** [installed_h5py] Error 2
make: *** [installed_pylith] Error 2

To ensure we use the correction Python for running gmsh (python may not exist, we need to change the #!/usr/bin/env python to #!/usr/bin/env nemesis, like we do for all Python scripts in the binary package.

OS X El Capitan removed support for DYLD_LIBRARY_PATH. As a result we use otool to generate link paths relative to the executable. We currently use the Python update_darwinlinking.py script to do this manually after running the packager/build.py script.

We should integrate this updating of link paths into the build.py script.

Note: We must update the link paths for all module .so files within the Python eggs as well.

Level of difficulty: Easy; this is a straightforward integration of one Python script into another plus some additional automation of shell commands.

PETSc does not get installed correctly when using the v3.0.3-0 installer. make install and make check are not getting run.

The fix is to have the installed_petsc target make both build_petsc and petsc rather than having make build_petsc as part of the petsc target.

Packages added for master / v2.2.1rc1 fail.

• HDF5 v1.10.2 build fails for some clang versions
• NetCDF v.4.6.1 build fails for some clang versions
• Need to update OpenMPI and test v3.1.0.

Need to pinpoint under what conditions these build failures occur.

Comment from Josimar regarding building v2.2.1 from source and trying to build the openssl libraries.

I solved this problem. Fundamentally, in my case, during the building process the openssl libraries were not being linked at all during the python installation part. Thus, I modified the existing pylith 2.2.1 Makefile so that it finds the correct openssl libraries.

In summary, if anyone else finds similar issues in the future, here are the steps:

1. Install openssl (I used homebrew)

2. Run configure as normal for the pylith installation

3. Before running "make", open the Makefile and add two extra paths to the LDFLAGS and CPPFLAGS. For my case, I had to add:
   a) LDFLAGS: -L/usr/local/opt/openssl/lib
   b) CPPFLAGS: -I/usr/local/opt/openssl/include

4. I am under the impression that the default path for pylith to look for openssl is /usr/local/ssl, which in my case does not exist.

Note that simply configuring pylith-2.2.1 using enable-openssl did not solve my problem.

Build PyLith Linux binaries using oldest LTS CentOS version for maximum compatibility. CentOS glibc version lags behind other Linux distributions (Debian, etc).

Installation of the Proj.4 datums requires unzip. When installing Proj.4, the configure script should check for unzip.

• Ubuntu 20.04 (Python 3.8)
• Ubuntu 22.04 (Python 3.10)
• Ubuntu 23.04 (Python 3.11)
• Debian stable (Python 3.9)
• Debian testing (Python 3.11)
• Fedora 36 (Python 3.10)
• Fedora 37 (Python 3.11)
• Update Makefile for building Proj 9 (uses CMake)
• RockyLinux 8 (Python 3.9)
• RockyLinux 9 (Python 3.9)
• CentOS 7

The installer uses the compiler specified by CC to build Python. The installer will detect and use mpicc and mpicxx for building anything that needs MPI.

If a user configures the installer with CC=mpicc CXX=mpicxx, it will generally cause problems building Python because the Python configure script includes:

# ICC needs -fp-model strict or floats behave badly
case "$CC" in
*icc*)
    BASECFLAGS="$BASECFLAGS -fp-model strict"
    ;;
esac

As a result, the Python configure thinks it is being built with an Intel compiler and it adds -fp-model strict to the CFLAGS.

The solution is to configure the installer with CC=NON_MPI_C_COMPILER and CXX=NON_MPI_C++_COMPILER. For example, CC=gcc CXX=g++ and let the installer detect mpicc and mpicxx that are on the path for building anything that needs MPI.

Python3 includes venv for virtual environments. We should add a configure option to create a virtual environment (default=yes, which would create a virtual environment in the installation directory).

For CIG codes (pythia, spatialdata, petsc, and PyLith) split installed_PACKAGE into

• fetch-PACKAGE
• configure-PACKAGE
• build-PACKAGE
• install-PACKAGE
• check-PACKAGE

This would permit user/developers to run just the steps they need after modifying the code.