Comments (3)
For example, with this setup 1st party code:
/tmp/example $ tree
.
└── package
├── bar.py
├── baz.py
└── foo.py
1 directory, 3 files
/tmp/example $ cat package/foo.py
from package import bar, baz
if __name__ == "__main__":
print(
"3rdparty paths:\n{paths}".format(paths="\n".join((bar.colors_path(), baz.cowsay_path())))
)
/tmp/example $ cat package/bar.py
import colors
def colors_path():
return colors.__file__
/tmp/example $ cat package/baz.py
import cowsay
def cowsay_path():
return cowsay.__file__
I can try to run it and it fails since I don't have the 3rdparty requirements:
/tmp/example $ python -m package.foo
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/tmp/example/package/foo.py", line 1, in <module>
from package import bar, baz
File "/tmp/example/package/bar.py", line 1, in <module>
import colors
ModuleNotFoundError: No module named 'colors'
But, I can create a 3rdparty dependency PEX and use it like a python interpreter:
/tmp/example $ pex ansicolors cowsay -o 3rdparty.pex
/tmp/example $ ./3rdparty.pex -m package.foo
3rdparty paths:
/home/jsirois/.pex/installed_wheels/00d2dde5a675579325902536738dd27e4fac1fd68f773fe36c21044eb559e187/ansicolors-1.1.8-py2.py3-none-any.whl/colors/__init__.py
/home/jsirois/.pex/installed_wheels/274b1e6fc1b966d53976333eb90ac94cb07a450a700b455af9fbdf882244b30a/cowsay-6.1-py3-none-any.whl/cowsay/__init__.py
So, here, just as with a normal python interpreter, the $PWD
is on the sys.path
and the PEX can see both the 1st party code in PWD
and the 3rdparty dependencies in the PEX file and everything works.
You can also explicitly steer the PEX to find the 1st party code:
# Can't find it.
/tmp/example $ mv 3rdparty.pex ..
/tmp $ ./3rdparty.pex -m package.foo
...
File "/home/jsirois/.pex/unzipped_pexes/c144d47a2d979a2605964d3858d98aaf6adf9f6f/.bootstrap/pex/pex.py", line 818, in execute_module
runpy.run_module(module_name, run_name="__main__", alter_sys=True)
File "<frozen runpy>", line 222, in run_module
File "<frozen runpy>", line 140, in _get_module_details
ImportError: Error while finding module specification for 'package.foo' (ModuleNotFoundError: No module named 'package')
# Still can't find it because PEXes are hermetic by default and ignore your environment (the `PYTHONPATH` setting in this case).
$ PYTHONPATH=example ./3rdparty.pex -m package.foo
File "/home/jsirois/.pex/unzipped_pexes/c144d47a2d979a2605964d3858d98aaf6adf9f6f/.bootstrap/pex/pex.py", line 818, in execute_module
runpy.run_module(module_name, run_name="__main__", alter_sys=True)
File "<frozen runpy>", line 222, in run_module
File "<frozen runpy>", line 140, in _get_module_details
ImportError: Error while finding module specification for 'package.foo' (ModuleNotFoundError: No module named 'package')
# Now can find it.
$ PEX_INHERIT_PATH=prefer PYTHONPATH=example ./3rdparty.pex -m package.foo
3rdparty paths:
/home/jsirois/.pex/installed_wheels/00d2dde5a675579325902536738dd27e4fac1fd68f773fe36c21044eb559e187/ansicolors-1.1.8-py2.py3-none-any.whl/colors/__init__.py
/home/jsirois/.pex/installed_wheels/274b1e6fc1b966d53976333eb90ac94cb07a450a700b455af9fbdf882244b30a/cowsay-6.1-py3-none-any.whl/cowsay/__init__.py
In short there are many ways to separate 1st and 3rdparty code in PEXes.
You should read up on both the output of pex --help
and pex --help-variables
now that you know about the "inherit path" option. Another similar runtime option is PEX_EXTRA_SYS_PATH
.
from pex.
@eci-aashish I've labelled this as a question and would like to close as answered if you can confirm either the --inherit-path
PEX build-time option or PEX_INHERIT_PATH
runtime equivalent help you out.
from pex.
@eci-aashish I gave you a good amount of detail there to work with. I'm going to assume the silence means your question is answered and close. Speak up if not.
from pex.
Related Issues (20)
- ValueError: invalid literal for int() with base 10 during pex build HOT 5
- How to bootstrap external pex files within running Python interpreter HOT 23
- Finalize Python 3.13 support HOT 1
- Build fails/incomplete on requirements.txt schema not followed correctly HOT 9
- Allow subsetting a PEX lockfile in PEX format, not just pip HOT 12
- Support for pip-compile generated requirements with hashes HOT 8
- bdist_pex called from tox fails with python 3.12 HOT 7
- Regression of artifact URL due to change since pip 23.3 HOT 12
- Monkey patch warning when using PEX to run a gunicorn app HOT 6
- `pex3 lock export` can turn VCS requirements into PyPI ones, resulting in hash failures HOT 3
- A way to configure sys.executable flags at PEX runtime HOT 8
- Add support for `--override`ing transitive dependencies. HOT 1
- Add `--provided`: like `--exclude`, but require the excluded project is available at boot time.
- "Resolving wheels disallowed" error when using `--lock` and `--only-binary` HOT 3
- Add more distribution metadata parsing context to failures.
- Add support for Pip 24.2 / statically linked musl libc CPythons
- Adding support to have all command line flags respected via environment variables HOT 6
- How does pex resolve defaut `--pip-version` HOT 13
- Support `--scie-pbs-flavor`. HOT 3
- Support `--scie` for PyPy.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pex.