Giter VIP home page Giter VIP logo

Comments (3)

jsirois avatar jsirois commented on August 13, 2024

For example, with this setup 1st party code:

/tmp/example $ tree
.
└── package
    ├── bar.py
    ├── baz.py
    └── foo.py

1 directory, 3 files
/tmp/example $ cat package/foo.py
from package import bar, baz


if __name__ == "__main__":
    print(
        "3rdparty paths:\n{paths}".format(paths="\n".join((bar.colors_path(), baz.cowsay_path())))
    )

/tmp/example $ cat package/bar.py
import colors


def colors_path():
    return colors.__file__

/tmp/example $ cat package/baz.py
import cowsay


def cowsay_path():
    return cowsay.__file__

I can try to run it and it fails since I don't have the 3rdparty requirements:

/tmp/example $ python -m package.foo
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/tmp/example/package/foo.py", line 1, in <module>
    from package import bar, baz
  File "/tmp/example/package/bar.py", line 1, in <module>
    import colors
ModuleNotFoundError: No module named 'colors'

But, I can create a 3rdparty dependency PEX and use it like a python interpreter:

/tmp/example $ pex ansicolors cowsay -o 3rdparty.pex
/tmp/example $ ./3rdparty.pex -m package.foo
3rdparty paths:
/home/jsirois/.pex/installed_wheels/00d2dde5a675579325902536738dd27e4fac1fd68f773fe36c21044eb559e187/ansicolors-1.1.8-py2.py3-none-any.whl/colors/__init__.py
/home/jsirois/.pex/installed_wheels/274b1e6fc1b966d53976333eb90ac94cb07a450a700b455af9fbdf882244b30a/cowsay-6.1-py3-none-any.whl/cowsay/__init__.py

So, here, just as with a normal python interpreter, the $PWD is on the sys.path and the PEX can see both the 1st party code in PWD and the 3rdparty dependencies in the PEX file and everything works.

You can also explicitly steer the PEX to find the 1st party code:

# Can't find it.
/tmp/example $ mv 3rdparty.pex ..
/tmp $ ./3rdparty.pex -m package.foo
...
  File "/home/jsirois/.pex/unzipped_pexes/c144d47a2d979a2605964d3858d98aaf6adf9f6f/.bootstrap/pex/pex.py", line 818, in execute_module
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "<frozen runpy>", line 222, in run_module
  File "<frozen runpy>", line 140, in _get_module_details
ImportError: Error while finding module specification for 'package.foo' (ModuleNotFoundError: No module named 'package')

# Still can't find it because PEXes are hermetic by default and ignore your environment (the `PYTHONPATH` setting in this case).
$ PYTHONPATH=example ./3rdparty.pex -m package.foo
  File "/home/jsirois/.pex/unzipped_pexes/c144d47a2d979a2605964d3858d98aaf6adf9f6f/.bootstrap/pex/pex.py", line 818, in execute_module
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "<frozen runpy>", line 222, in run_module
  File "<frozen runpy>", line 140, in _get_module_details
ImportError: Error while finding module specification for 'package.foo' (ModuleNotFoundError: No module named 'package')

# Now can find it.
$ PEX_INHERIT_PATH=prefer PYTHONPATH=example ./3rdparty.pex -m package.foo
3rdparty paths:
/home/jsirois/.pex/installed_wheels/00d2dde5a675579325902536738dd27e4fac1fd68f773fe36c21044eb559e187/ansicolors-1.1.8-py2.py3-none-any.whl/colors/__init__.py
/home/jsirois/.pex/installed_wheels/274b1e6fc1b966d53976333eb90ac94cb07a450a700b455af9fbdf882244b30a/cowsay-6.1-py3-none-any.whl/cowsay/__init__.py

In short there are many ways to separate 1st and 3rdparty code in PEXes.

You should read up on both the output of pex --help and pex --help-variables now that you know about the "inherit path" option. Another similar runtime option is PEX_EXTRA_SYS_PATH.

from pex.

jsirois avatar jsirois commented on August 13, 2024

@eci-aashish I've labelled this as a question and would like to close as answered if you can confirm either the --inherit-path PEX build-time option or PEX_INHERIT_PATH runtime equivalent help you out.

from pex.

jsirois avatar jsirois commented on August 13, 2024

@eci-aashish I gave you a good amount of detail there to work with. I'm going to assume the silence means your question is answered and close. Speak up if not.

from pex.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.