saltstack / relenv Goto Github PK

Recent versions of relenv (at least 0.15.0, 0.15.1, and 0.16.0) break pip install when used with the --target argument.

This causes salt-pip not to work correctly when trying to install some Python modules like mysqlclient (saltstack/salt#65980) or M2Crypto (saltstack/salt#66311).

I can definitely track this down to an issue in relenv because the problem can be resolved by commenting removing a single line in relenv’s runtime.py:

--- relenv/runtime.py.orig	2024-05-17 14:49:35.522975150 +0000
+++ relenv/runtime.py	2024-05-17 14:51:00.735990855 +0000
@@ -783,7 +783,6 @@
         Wrapper("pip._internal.operations.install.wheel", wrap_pip_install_wheel),
         Wrapper("pip._internal.operations.install.legacy", wrap_pip_install_legacy),
         Wrapper("pip._internal.operations.build.wheel", wrap_pip_build_wheel),
-        Wrapper("pip._internal.commands.install", wrap_cmd_install),
         Wrapper("pip._internal.locations", wrap_locations),
         Wrapper("pip._internal.cli.req_command", wrap_req_command),
         Wrapper("pip._internal.req.req_install", wrap_req_install),

As this might have undesired side effects (I do not understand the code well enough to be sure): I also tried more localized changes and came up with the following patch that resolves the issue:

--- relenv/runtime.py.orig	2024-05-17 14:49:35.522975150 +0000
+++ relenv/runtime.py	2024-05-17 14:53:50.545215677 +0000
@@ -641,19 +641,6 @@
 
     mod.InstallCommand.run = wrap(mod.InstallCommand.run)
 
-    def wrap(func):
-        @functools.wraps(func)
-        def wrapper(self, target_dir, target_temp_dir, upgrade):
-            from pip._internal.cli.status_codes import SUCCESS
-
-            return SUCCESS
-
-        return wrapper
-
-    if hasattr(mod.InstallCommand, "_handle_target_dir"):
-        mod.InstallCommand._handle_target_dir = wrap(
-            mod.InstallCommand._handle_target_dir
-        )
     return mod
 
 
@@ -670,8 +657,6 @@
         ):
             scheme = func(dist_name, user, home, root, isolated, prefix)
             if TARGET.TARGET:
-                scheme.platlib = TARGET.PATH
-                scheme.purelib = TARGET.PATH
                 scheme.data = TARGET.PATH
             return scheme
 
@@ -729,8 +714,6 @@
                 use_user_site=False,
                 pycompile=True,
             ):
-                if TARGET.TARGET:
-                    home = TARGET.PATH
                 return func(
                     self,
                     install_options,
@@ -756,8 +739,6 @@
                 use_user_site=False,
                 pycompile=True,
             ):
-                if TARGET.TARGET:
-                    home = TARGET.PATH
                 return func(
                     self,
                     global_options,

In order to resolve the error when trying to install the module, the change that avoids setting scheme.platlib and scheme.purelib is sufficient, but when only applying this change and not the other changes, the installed modules end up in /opt/saltstack/salt/extras-3.10/lib/python instead of /opt/saltstack/salt/extras-3.10.

When applying the change regarding scheme.purelib and home = TARGET.PATH, but not removing the wrapper for _handle_target_dir, there is no error but the module is not installed at all.

If I understand it correctly, home is usually set to a temporary directory and _handle_target_dir copies it from there to the actual target path. This is why not overriding home but replacing _handle_target_dir results in the module not being installed at all (it ends up in the temporary directory and is never moded from there).

@dwoz Unfortunately, I do not understand the intention behind this code, so I am not sure whether my fix is correct or might break something else. As it seems like you wrote most of this code, I hope that you might be able to explain what this code is supposed to do.

Maybe this code was targeted at an older version of pip, and due to changes in pip, it does not work correctly any longer (after all, the code messes with internals of pip). I tested my patch with Salt 3006.8, which bundles pip 23.3.2.

Some of the commits that added the code that is removed by my patch say something about fixing pip uninstall, but at fore the aforementioned version of pip, uninstall still works correctly after applying my patch.

Defining the RELENV_PIP_DIR environment variable should cause pip to install scripts into the root the environment instead of the bin directory. This functionality works on linux but not on windows. I suspect this has to do with the version of python on windows and not something platform specific.

When I try to run pip commands I get the following:

Salts-Mac:osx saltadmin$ /Users/saltadmin/src/salt/pkg/osx/build/opt/salt/bin/pip3 list
readlink: illegal option -- f
usage: readlink [-n] [file ...]
usage: dirname path
/Users/saltadmin/src/salt/pkg/osx/build/opt/salt/bin/pip3: line 2: /python3: No such file or directory
/Users/saltadmin/src/salt/pkg/osx/build/opt/salt/bin/pip3: line 2: exec: /python3: cannot execute: No such file or directory

Running pip from python works:

Salts-Mac:osx saltadmin$ /Users/saltadmin/src/salt/pkg/osx/build/opt/salt/bin/python3 -m pip list
Package    Version
---------- -------
pip        22.3.1
relenv     0.4.9
setuptools 65.5.0
wheel      0.38.4

Installing packages with salt-pip (Salt 3006.0) fails with a JSONDecodeError and what appears to be some recursive behavior.

To reproduce, set up a fresh Almalinux 8.7 box (I use Vagrant on QEMU) and manually install the Salt 3006.0 minion. Add a symlink according to #114:

ln -s /opt/saltstack/salt/bin/python3 /usr/bin/python3

Then, attempt to install pycryptodome (or any other package in my experience):

/opt/saltstack/salt/salt-pip install pycryptodome

Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/saltstack/salt/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/__main__.py", line 29, in <module>
    from pip._internal.cli.main import main as _main
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/main.py", line 9, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/main_parser.py", line 9, in <module>
    from pip._internal.build_env import get_runnable_pip
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/build_env.py", line 20, in <module>
    from pip._internal.cli.spinners import open_spinner
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/spinners.py", line 9, in <module>
    from pip._internal.utils.logging import get_indentation
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/utils/logging.py", line 29, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 42, in <module>
    from pip._internal.locations import get_major_minor_version
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/locations/__init__.py", line 14, in <module>
    from . import _sysconfig
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/locations/_sysconfig.py", line 11, in <module>
    from .base import change_root, get_major_minor_version, is_osx_framework
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/locations/base.py", line 16, in <module>
    site_packages: typing.Optional[str] = sysconfig.get_path("purelib")
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 572, in get_path
    return get_paths(scheme, vars, expand)[name]
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 167, in wrapped
    paths = func(scheme=scheme, vars=vars, expand=expand)
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 562, in get_paths
    return _expand_vars(scheme, vars)
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 218, in _expand_vars
    _extend_dict(vars, get_config_vars())
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 131, in wrapped
    _CONFIG_VARS = func()
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 640, in get_config_vars
    srcdir = os.path.dirname(get_makefile_filename())
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 399, in get_makefile_filename
    return os.path.join(get_path('stdlib'), config_dir_name, 'Makefile')
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 572, in get_path
    return get_paths(scheme, vars, expand)[name]
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 167, in wrapped
    paths = func(scheme=scheme, vars=vars, expand=expand)
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 562, in get_paths
    return _expand_vars(scheme, vars)
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 218, in _expand_vars
    _extend_dict(vars, get_config_vars())
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 140, in wrapped
    _SYSTEM_CONFIG_VARS = json.loads(p.stdout[:-1])
  File "/opt/saltstack/salt/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/opt/saltstack/salt/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/opt/saltstack/salt/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The JSONDecodeError is due to get_config_vars_wrapper(...) parsing the stdout of a subprocess Python call to sysconfig: in my environment, calling sysconfig.get_config_vars() actually prints nothing (stdout empty), causing JSON decoding to fail. (Side note: did Rube Goldberg write this function? I'm sure there's a reason for the way this is implemented; could someone elaborate?)

Digging deeper into sysconfig.get_config_vars() reveals some apparently recursive behavior. The following is what I have observed in my environment when calling sysconfig.get_config_vars() in the same manner as get_config_vars_wrapper() from Salt 3006.0's Python 3.10 REPL:

1. sysconfig.get_config_vars() is called without arguments. _CONFIG_VARS is None, the os.name is posix and _PYTHON_BUILD is falsy. Execution continues to get_makefile_filename().
2. Execution continues to a call to get_path('stdlib').
3. get_paths('posix_prefix', None, True) is called. It in turn calls _expand_vars('posix_prefix', None).
4. _expand_vars calls _extend_dict(None, get_config_vars()). Notice that get_config_vars() is now being recursively called.
5. At this point in practice, the wrapper is called instead of get_config_vars. The wrapper, again, calls sysconfig.get_config_vars(None) within a new Python subprocess, and this time _CONFIG_VARS has a bunch (722 for me) of variables, which it returns.
6. For some reason I haven't wrapped my mind around, the now non-empty variables result is not returned from the wrapper back to _extend_dict. In fact, _extend_dict is never called. Instead, the wrapper recursively calls itself. I've seen it do so up to 1,024 times, and have heard from another user with this problem that it is accompanied by a spike in memory and crash of the Salt minion and master.

Interestingly, this behavior seems to only occur when using Salt's bundled Python at /usr/bin/python3. Compiling Python 3.10.11 from source and adding the appropriate symlink for that scenario (ln -s /usr/local/bin/python3 /usr/bin/python3) does not present this problem.

Firstly, I've been watching relenv for a while - I think it's rad. Great work 😎

I would like to add support for customizing the build environment (e.g. what's created with relenv build). Specifically:

• Custom [Cc]onfigure flags for CPython, OpenSSL, SQLite, etc.
• Custom CFLAGS / LDFLAGS / CXXFLAGS
• Other Custom environment variables (e.g. MACOSX_DEPLOYMENT_TARGET)
• Make flags
• Customization of source tarballs (Download URL's, Versions, Checksums)

Although my primary focus is on macOS (as I support my Company's fleet of macOS devices, including its use of Salt), I think customization would be valuable for all platforms.

As a good example, take issue #114, where the default /usr/bin/python3 executable is hard-coded in relenv, and is actually immutable on macOS (due to System Integrity Protection), but can "point" to different Xcode environments (via xcode-select).

If I wanted to use zlib or libzmq from Homebrew (instead of building it myself), currently relenv doesn't allow me to do that.

I want to be able use a different Python version (e.g. a pre-release version, 3.9.x, or any other version not specifically hard-coded into relenv.

Would be nice to be able to suppress the output if needed

When attempting to install salt using the onedir method I encountered a failure where some imports in the relenv Python environment could not be found (ex. runpy). After doing some digging, I was able to identify that the mechanism relenv uses to sanitize sys.path will remove valid entries if the Python interpreter is called from a path that contains a symlink anywhere along the path.

Steps to Reproduce

1. Create a target directory and link it to a directory called parent: mkdir -p target/ && ln -sf target/ parent/.
2. Create a new relenv inside the parent directory: cd parent/ && relenv create testenv.
3. Go up one directory and launch the python interpreter inside the relenv with debugging enabled and observe that sys.path has been cleared.

$ RELENV_DEBUG=true parent/testenv/bin/python3
Unable to get the modules directory from openssl: version: Option unknown option -m
version: Use -help for summary.

original sys.path was ['/root/parent/testenv/lib/python310.zip', '/root/parent/testenv/lib/python3.10', '/root/parent/testenv/lib/python3.10/lib-dynload', '/root/parent/testenv/lib/python3.10/site-packages']
new sys.path is []
After site customize wrapper
Python 3.10.13 (main, Nov  8 2023, 12:14:32) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Root Cause

After digging into the code, I identified that the sanitize_sys_path function incorrectly removes entries in sys.path because it doesn't resolve symlinks properly. Specifically this code:

https://github.com/saltstack/relative-environment-for-python/blob/78b8dbf8d5d76612bc79e8ab8760ffc82223e072/relenv/common.py#L471-L472

A call to resolve() needs to happen before checking if the sys.path entry is relative to the valid prefix. Something like the following would resolve this issue.

__resolved_path = pathlib.Path(__path).resolve()
__resolved_path.relative_to(__valid_path_prefix)
__sys_path.append(str(__resolved_path))

Currently the lack of proper path resolving breaks installing salt in onedir mode if /opt is symlinked to another directory.

Looks like docs have not been updated during the last few releases. We need to investigate why docs were not publish and If docs fail the build should fail.

Currently we're cross compiling python aarch64 for linux on x86_64.

Add a build which builds aarch64 on aarch64 to make sure we're testing this case for now.

We need to be able to build 32-bit Python on Windows. As per the output of --help the only options are x86_64 and aarch64:

C:\src\Mayflower>python -m mayflower build --help
usage: __main__.py build [-h] [--arch {x86_64,aarch64}] [--clean] [--no-cleanup] [--no-download] [--step STEP]

Build Mayflower Python Environments from source

optional arguments:
  -h, --help            show this help message and exit
  --arch {x86_64,aarch64}
                        The host architecture [default: x86_64]
  --clean               Clean up before running the build. This option will remove the logs, src, build, and previous tarball.
  --no-cleanup          By default the build directory is removed after the build tarball is created. Setting this option will leave the build directory in place.
  --no-download         Skip downloading source tarballs
  --step STEP           A step to run alone, can use multiple of this argument. When this option is used to invoke builds, depenencies of the steps are ignored. This option should be used with care, as it's easy to request a situation that has no chance of
                        being succesful.```

The c-extensions built by newer cryptography versions are not getting the proper rpath set.

Looking through the long list of bundled dependencies, I'm slightly concerned about how old some of the software releases are, especially given that the whole point of this tool is to have zero dependencies. Some are somewhat unimportant/irrelevant, but others like libssl and python are frankly quite important

Looking through the bundled software (for Linux at least; it's the only platform that interests me) here https://github.com/saltstack/relative-environment-for-python/blob/b848c6f7cf7a36587935db349c42e89e0fb0e5b6/relenv/build/linux.py#L353

From what I can see, this doesn't even have the excuse of "it was latest when we started" because many of these "latest" versions even predate the initial commit of this data: b48d9ca

I would consider these issues above significant enough to not consider using relenv-based Salt packages until these have been addressed. To me, the whole (and only) reason to go to all of the effort to build what is effectively an isolated system of libraries (like you would in a container) like this is to be able to break away from the grasp of "stable" distributions with old versions and to be able to use new features, bug/security fixes and performance enhancements. Instead, what relenv provides currently is a liability to fork, build and maintain newer versions of libraries to stay secure, but no perceivable benefits for the user

In the following example scripts will be missing.

myrelenv/bin/pip install --target=myrelenv/extras rend
myrelenv/bin/pip install --target=myrelenv/extras cowsay

The cowsay script is missing from myrelenv/extras/bin

See also saltstack/salt#64662

There should be a way to detect if the python being used is a relenv onedir or not.

Checking if relenv is installed is not enough. Relenv can be installed into python environments to bootstrap onedir environments, in fact, that's how relenv works.

So, how can we detect?

Python has built-in support for frozen(sys.frozen). PyInstaller uses sys.MEIPASS.

Set sys.relenv = True when relenv is bootstrapped at runtime?

When using pip with --target the shebangs in python scripts created by pip are broken.

Since about 20 June, some tests and installations of Ubuntu machines involving the pip.installed command have been failing. The problem's appearance coincided with the appearance of pip version 24.1, but the affected Ubuntu machines have version 22.0.2. The affected Ubuntu machines are using relenv version 0.16.0, Python 3.10.14 and salt-minion 3006.8.

The error appears in this way:

     ID: pip                                                                (This part is okay)

Function: pip.installed
Result: True
Comment: All packages were successfully installed
Started: 11:48:07.543580
Duration: 9436.212 ms
Changes:
----------
pip==24.1:
Installed

     ID: pyinotify

Function: pip.installed
Result: False
Comment: Failed to install packages: pyinotify. Error: Looking in indexes: (index removed)
Collecting pyinotify
Downloading (URL removed) (60 kB)
>���������������������������������������� 61.0/61.0 kB 1.0 MB/s eta 0:00:00
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: pyinotify
Building wheel for pyinotify (setup.py): started
Building wheel for pyinotify (setup.py): finished with status 'done'
Created wheel for pyinotify: filename=pyinotify-0.9.6-py3-none-any.whl size=25342 sha256=2ed656274ed34a4ddf69abdcf7d167729fe972c03b6b4d900e5e1bb4fa8394b7
Stored in directory: /root/.cache/pip/wheels/4a/04/c7/8ad5f6b9280b1fe87df10fd6473716f4f1f6a922fbbfb94bc8
Successfully built pyinotify
Installing collected packages: pyinotify ERROR: Exception:
Traceback (most recent call last):
File "/opt/saltstack/salt/extras-3.10/pip/_internal/cli/base_command.py", line 179, in exc_logging_wrapper
status = run_func(*args)
File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 638, in wrapper
return func(self, options, args)
File "/opt/saltstack/salt/extras-3.10/pip/_internal/cli/req_command.py", line 67, in wrapper
return func(self, options, args)
File "/opt/saltstack/salt/extras-3.10/pip/_internal/commands/install.py", line 455, in run
installed = install_given_reqs(
File "/opt/saltstack/salt/extras-3.10/pip/_internal/req/init.py", line 70, in install_given_reqs
requirement.install(
File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 761, in wrapper
return func(
File "/opt/saltstack/salt/extras-3.10/pip/_internal/req/req_install.py", line 817, in install
scheme = get_scheme(
File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 673, in wrapper
scheme.platlib = TARGET.PATH
File "", line 4, in setattr
dataclasses.FrozenInstanceError: cannot assign to field 'platlib'
Started: 11:48:17.530499
Duration: 2453.652 ms
Changes:

As a result, inotify beacons cannot be configured. This apparently affects at least one other module in pip.install commands. So far, this has only been observed in tests simulating the installation (i.e. initial highstate) of Ubuntu machines and in Ubuntu machines installed after 20 June. Machines installed at or before this date seem unaffected, even when they also have relenv version 0.16.0, Python 3.10.14 as well as salt-minion 3006.8. It is known to affect Ubuntu 20.04 and 22.04. Earlier and later versions of Ubuntu were not tested.

Deleting line 673 of /opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py resulted in the dataclasses.FrozenInstanceError appearing for 'purelib' instead (line 674). Deleting this line too moved the dataclasses.FrozenInstanceError to 'data' (line 675). Deleting line 675 and the if statement in line 672 removed the error messages, but this naturally does not install the module.

Using sitecustomize.py to bootstrap relenv's runtime prevents other tools from leveraging sitecustomize.py.

The toolchains seem to have started failing to build because crosstool-ng is un-able to download zlib.

We discovered this when using the pre-built macos python build to make the Salt Mac package. It seems we're linking to gettext in /usr/local for some reason. When installing salt into the environment we get the following traceback.

dyld: Library not loaded: /usr/local/opt/gettext/lib/libintl.8.dylib
  Referenced from: /Users/saltadmin/src/salt/pkg/osx/build/opt/salt/bin/python3
  Reason: image not found
/Users/saltadmin/src/salt/pkg/osx/install_salt.sh: line 102:   968 Abort trap: 6   

Everything works when using relenv build to build the python on the macos box where this was tested.

Currently, the version of Python built by Mayflower is hard-coded. We need the ability to pass a version of Python to build.

Using the system python

No virtualenv:

Python 3.11.0 (main, Sep 12 2023, 18:26:57) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.md5()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: [digital envelope routines] unsupported
>>> hashlib.sha256()
<sha256 _hashlib.HASH object @ 0x7f030577b7d0>
>>>

Virtualenv created using the system python:

root@ip-10-2-3-37 [ ~ ]# python3 -m virtualenv /tmp/ve1
created virtual environment CPython3.11.0.final.0-64 in 346ms
  creator CPython3Posix(dest=/tmp/ve1, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.2.2, wheel==0.41.2
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
root@ip-10-2-3-37 [ ~ ]# source /tmp/ve1/bin/activate

(ve1) root@ip-10-2-3-37 [ ~ ]# python
Python 3.11.0 (main, Sep 12 2023, 18:26:57) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.md5()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: [digital envelope routines] unsupported
>>> hashlib.sha256()
<sha256 _hashlib.HASH object @ 0x7f4c324e6670>
>>>

Now using relenv

No virtualenv

root@ip-10-2-3-37 [ ~ ]# /tmp/testing/artifacts/salt/bin/python3
Python 3.10.13 (main, Sep  9 2023, 07:21:14) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.md5()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: [digital envelope routines] unsupported
>>> hashlib.sha256()
<sha256 _hashlib.HASH object @ 0x7f0b18d01d50>
>>>

A virtualenv using relenv

root@ip-10-2-3-37 [ ~ ]# /tmp/testing/artifacts/salt/bin/python3 -m pip install virtualenv
Collecting virtualenv
  Downloading virtualenv-20.24.5-py3-none-any.whl (3.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.7/3.7 MB 14.6 MB/s eta 0:00:00
Collecting filelock<4,>=3.12.2
  Downloading filelock-3.12.4-py3-none-any.whl (11 kB)
Collecting platformdirs<4,>=3.9.1
  Downloading platformdirs-3.11.0-py3-none-any.whl (17 kB)
Collecting distlib<1,>=0.3.7
  Downloading distlib-0.3.7-py2.py3-none-any.whl (468 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 468.9/468.9 kB 30.3 MB/s eta 0:00:00
Installing collected packages: distlib, platformdirs, filelock, virtualenv
Successfully installed distlib-0.3.7 filelock-3.12.4 platformdirs-3.11.0 virtualenv-20.24.5
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv

[notice] A new release of pip available: 22.3.1 -> 23.3.1
[notice] To update, run: /tmp/testing/artifacts/salt/bin/python3 -m pip install --upgrade pip
root@ip-10-2-3-37 [ ~ ]# /tmp/testing/artifacts/salt/bin/python3 -m virtualenv /tmp/ve2
created virtual environment CPython3.10.13.final.0-64 in 314ms
  creator CPython3Posix(dest=/tmp/ve2, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==23.2.1, setuptools==68.2.2, wheel==0.41.2
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
root@ip-10-2-3-37 [ ~ ]# source /tmp/ve2/bin/activate

(ve2) root@ip-10-2-3-37 [ ~ ]# python
Python 3.10.13 (main, Sep  9 2023, 07:21:14) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import hashlib
>>> hashlib.md5()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: [digital envelope routines] unsupported
>>> hashlib.sha256()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: [digital envelope routines] unsupported
>>>

Refs saltstack/salt#65444

Add two more types of keys. In addition to minion; add master and client (cli) keys.

Installing Pycurl needs a libcurl that is compiled against the proper glibc version. Document the process of building libcurl to work with relenv.

A couple warnings started popping up.

2023-08-28T21:30:28.2106254Z <frozen importlib._bootstrap>:283: DeprecationWarning: the load_module() method is deprecated and slated for removal in Python 3.12; use exec_module() instead
2023-08-28T21:30:28.8675492Z <frozen importlib._bootstrap>:914: ImportWarning: RelenvImporter.find_spec() not found; falling back to find_module()

get_config_vars_wrapper(...) runs Python as a subprocess (more explanation on why this is necessary would be great). The path to Python to run in the subprocess is hard coded to /usr/bin/python3, which is not always installed and would require either a symlink to Salt's salt-pip or a system Python installation.

To reproduce, spin up Almalinux 8.7 (I do so in Vagrant on QEMU) and manually install the Salt minion, confirming there is no Python installed on the system. Then, log in and execute:

/opt/saltstack/salt/salt-pip install pycryptodome

Traceback (most recent call last):
  File "/opt/saltstack/salt/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/saltstack/salt/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/__main__.py", line 29, in <module>
    from pip._internal.cli.main import main as _main
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/main.py", line 9, in <module>
    from pip._internal.cli.autocompletion import autocomplete
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/autocompletion.py", line 10, in <module>
    from pip._internal.cli.main_parser import create_main_parser
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/main_parser.py", line 9, in <module>
    from pip._internal.build_env import get_runnable_pip
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/build_env.py", line 20, in <module>
    from pip._internal.cli.spinners import open_spinner
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/cli/spinners.py", line 9, in <module>
    from pip._internal.utils.logging import get_indentation
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/utils/logging.py", line 29, in <module>
    from pip._internal.utils.misc import ensure_dir
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/utils/misc.py", line 42, in <module>
    from pip._internal.locations import get_major_minor_version
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/locations/__init__.py", line 14, in <module>
    from . import _sysconfig
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/locations/_sysconfig.py", line 11, in <module>
    from .base import change_root, get_major_minor_version, is_osx_framework
  File "/opt/saltstack/salt/lib/python3.10/site-packages/pip/_internal/locations/base.py", line 16, in <module>
    site_packages: typing.Optional[str] = sysconfig.get_path("purelib")
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 572, in get_path
    return get_paths(scheme, vars, expand)[name]
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 167, in wrapped
    paths = func(scheme=scheme, vars=vars, expand=expand)
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 562, in get_paths
    return _expand_vars(scheme, vars)
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 218, in _expand_vars
    _extend_dict(vars, get_config_vars())
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 131, in wrapped
    _CONFIG_VARS = func()
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 640, in get_config_vars
    srcdir = os.path.dirname(get_makefile_filename())
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 399, in get_makefile_filename
    return os.path.join(get_path('stdlib'), config_dir_name, 'Makefile')
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 572, in get_path
    return get_paths(scheme, vars, expand)[name]
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 167, in wrapped
    paths = func(scheme=scheme, vars=vars, expand=expand)
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 562, in get_paths
    return _expand_vars(scheme, vars)
  File "/opt/saltstack/salt/lib/python3.10/sysconfig.py", line 218, in _expand_vars
    _extend_dict(vars, get_config_vars())
  File "/opt/saltstack/salt/lib/python3.10/site-packages/relenv/runtime.py", line 132, in wrapped
    p = subprocess.run(
  File "/opt/saltstack/salt/lib/python3.10/subprocess.py", line 503, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/opt/saltstack/salt/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/saltstack/salt/lib/python3.10/subprocess.py", line 1863, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/bin/python3'

This issue can be bypassed by:

• installing Python on the system
• creating a symlink:

  ln -s /opt/saltstack/salt/bin/python3 /usr/bin/python3

There should be better documentation about all of the things relenv does when python starts up.

Upgrade openssl to account for recent CVEs. Upgrade python versions. Also tackle any other dependencies that can be upgrades

Did relenv fetch, then relenv create. Result:

➜  ~ myenv/bin/python3 -V                
myenv/bin/python3: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory

We should validate the shebangs generated by relenv using the shellcheck tool.

saltstack/salt#63517

We need to add a step to the workflows to upload our toolchains and builds to repo.saltproject.io.

Salt 3006.5, Ubuntu 22.04:

salt-pip install debugpy
/opt/saltstack/salt/bin/python3 -c 'import debugpy; debugpy.listen(5678); debugpy.wait_for_client()'
0.69s - Error patching args (debugger not attached to subprocess).

Traceback (most recent call last):
  File "/opt/saltstack/salt/extras-3.10/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 526, in patch_args
    host, port = _get_host_port()
  File "/opt/saltstack/salt/extras-3.10/debugpy/_vendored/pydevd/_pydev_bundle/pydev_monkey.py", line 212, in _get_host_port
    host, port = pydevd.dispatch()
  File "/opt/saltstack/salt/extras-3.10/debugpy/_vendored/pydevd/pydevd.py", line 3084, in dispatch
    host = setup['client']
TypeError: 'NoneType' object is not subscriptable

The same command works with system Python binary.

relocate is not setting rpath properly on cross compiled builds. The root of the problem seems to be the cross toolchain's ldd is not returning the expected output.

Building M2Crypto for a relenv environment results in a segfault on import.

saltstack/salt#64288

Can we choose PYTHONUTF8=1 behavior to be the default on windows? This is when building python.

https://docs.python.org/3/using/windows.html#utf-8-mode

https://relenv.readthedocs.io/en/latest/toolchain.html references [email protected]:saltstack/relenv.git (by ssh, not https, too) which doesn't exist, and tells the user to use relenv toolchain build which doesn't exist:

(venv) frebib@frebib-PC /t/relenv.NCsd7Umb> python3.11 -m relenv toolchain download
usage: relenv toolchain [-h] [--arch {x86_64,aarch64}] [--clean] [--crosstool-only] {build,fetch}
relenv toolchain: error: argument command: invalid choice: 'download' (choose from 'build', 'fetch')

It would be nice to have the option to see the debug output to make debugging easier

Hashutils imported before ssl modules directory has been detected. The result is missing hashes when trying to use Openssl's legacy provider.

The 3006.2 release of Salt introduced a new version of relenv, which added code to runtime.py that sets OPENSSL_MODULES. It also added openssl modules to the relenv. However, this transparent environment mangling breaks (at least some) non-Salt tooling that uses OpenSSL.

For example, when using Salt to manage maas, we utilize a cmd.script state to run several shell commands that initialize a new maas instance, configure TLS, etc. With the latest changes to relenv released with Salt 3006.2, OPENSSL_MODULES is pointed at the OpenSSL within the relenv, and any commands that Salt runs inherit this modified environment. This means that, when Salt runs maas CLI commands, the maas CLI tools try (and fail) to load legacy.so, resulting in a cryptic OpenSSL error.

I was able to work around this by manually unsetting OPENSSL_MODULES in the shell script, but other tools are likely to be similarly affected.

Installing salt using USE_STATIC_REQUIREMENTS=1, which selects a different set of requirements, fails.

ubuntu@ip-10-1-102-50:~$ salt/bin/python3 -m pip install downloaded-requirements/cffi-1.14.6.tar.gz
Processing ./downloaded-requirements/cffi-1.14.6.tar.gz
  Preparing metadata (setup.py) ... done
Requirement already satisfied: pycparser in ./salt/lib/python3.10/site-packages (from cffi==1.14.6) (2.21)
Building wheels for collected packages: cffi
  Building wheel for cffi (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [38 lines of output]
      /home/ubuntu/salt/lib/python3.10/site-packages/setuptools/config/setupcfg.py:463: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-310
      creating build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/recompiler.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/vengine_cpy.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/backend_ctypes.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/setuptools_ext.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/error.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/pkgconfig.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/api.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/commontypes.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/model.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/cffi_opcode.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/vengine_gen.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/ffiplatform.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/verifier.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/__init__.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/cparser.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/lock.py -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/_cffi_include.h -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/parse_c_type.h -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/_embedding.h -> build/lib.linux-x86_64-cpython-310/cffi
      copying cffi/_cffi_errors.h -> build/lib.linux-x86_64-cpython-310/cffi
      running build_ext
      building '_cffi_backend' extension
      creating build/temp.linux-x86_64-cpython-310
      creating build/temp.linux-x86_64-cpython-310/c
      /home/ubuntu/.local/relenv/toolchain/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc -no-pie -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -L/home/ubuntu/.local/lib -L/home/ubuntu/.local/relenv/
toolchain/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/lib -I/home/ubuntu/.local/include -I/home/ubuntu/.local/include/readline -I/home/ubuntu/.local/include/ncursesw -I/home/ubuntu/.local/relenv/toolchain/x86_64-lin
ux-gnu/x86_64-linux-gnu/sysroot/usr/include -L/home/ubuntu/.local/lib -L/home/ubuntu/.local/relenv/toolchain/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/lib -I/home/ubuntu/.local/include -I/home/ubuntu/.local/includ
e/readline -I/home/ubuntu/.local/include/ncursesw -I/home/ubuntu/.local/relenv/toolchain/x86_64-linux-gnu/x86_64-linux-gnu/sysroot/usr/include -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/home/ubuntu/salt/includ
e/python3.10 -c c/_cffi_backend.c -o build/temp.linux-x86_64-cpython-310/c/_cffi_backend.o
      c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
         15 | #include <ffi.h>
            |          ^~~~~~~
      compilation terminated.
      error: command '/home/ubuntu/.local/relenv/toolchain/x86_64-linux-gnu/bin/x86_64-linux-gnu-gcc' failed with exit code 1

We see the following error on photon os with fips enabled. The fips module mac between the module shipped with photon and the module shipped with relenv differ. Rather than having to run relenv with it's own ssl config; default to the system's openssl modules location.

ValueError: [digital envelope routines] unsupported