Comments (8)
lorencarvalho
commented on August 26, 2024
Hi @rouge8,
Looks to be an known issue with Homebrew'd python: https://github.com/Homebrew/brew/blob/master/docs/Homebrew-and-Python.md#note-on-pip-install---user
Note that shiv is not using the --user command, but it does use --target which suffers the same fate.
You can see that plain old pip install exhibits the problem:
lcarvalh-mn1 ~ ❱❱❱ /usr/local/Cellar/python/3.6.3/bin/pip install --target temp_dir aws --no-deps
Collecting aws
Using cached https://files.pythonhosted.org/packages/90/56/d088e68699e4f00881d957584e26854a88d0009864ef614c6f567807fc2f/aws-0.2.5.tar.gz
Skipping bdist_wheel for aws, due to binaries being disabled for it.
Installing collected packages: aws
Running setup.py install for aws ... done
Exception:
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/commands/install.py", line 350, in run
isolated=options.isolated_mode,
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/commands/install.py", line 436, in get_lib_location_guesses
scheme = distutils_scheme('', *args, **kwargs)
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pip/locations.py", line 153, in distutils_scheme
i.finalize_options()
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/setuptools/command/install.py", line 38, in finalize_options
orig.install.finalize_options(self)
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/distutils/command/install.py", line 248, in finalize_options
"must supply either home or prefix/exec-prefix -- not both")
distutils.errors.DistutilsOptionError: must supply either home or prefix/exec-prefix -- not both
I tried one of the workarounds I found while googling around and it seemed to fix the problem:
lcarvalh-mn1 ~ ❱❱❱ cat << EOF > ~/.pydistutils.cfg
> [install]
> prefix=
> EOF
lcarvalh-mn1 ~ ❱❱❱ /usr/local/Cellar/python/3.6.3/bin/pip3 install --target temp_dir aws --no-deps
Collecting aws
Installing collected packages: aws
Successfully installed aws-0.2.5
However, you might still run into issues because that package (aws) appears to only support Python2 😞
lcarvalh-mn1 ~ ❱❱❱ shiv aws -c aws -p /usr/local/Cellar/python/3.6.3/bin/python3 -o blergh -q
lcarvalh-mn1 ~ ❱❱❱ ./blergh
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "./blergh/__main__.py", line 3, in <module>
File "./blergh/_bootstrap/__init__.py", line 102, in bootstrap
File "./blergh/_bootstrap/__init__.py", line 21, in import_string
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 941, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 994, in _gcd_import
File "<frozen importlib._bootstrap>", line 971, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 674, in exec_module
File "<frozen importlib._bootstrap_external>", line 781, in get_code
File "<frozen importlib._bootstrap_external>", line 741, in source_to_code
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/aws-0.2.5-py3.6.egg/aws/main.py", line 23
print '%(name)s: %(endpoint)s' % {
^
SyntaxError: invalid syntax
from shiv.
rouge8
commented on August 26, 2024
However, you might still run into issues because that package (aws) appears to only support Python2 😞
Ha, I meant to do awscli, but ran into that issue with a few packages. I'm guessing anything where a wheel is unavailable is affected.
I wonder if there's a workaround that can go into shiv because this is probably pretty common... I'll see if I can make anything work without needing a ~/.pydistutils.cf 😕
from shiv.
lorencarvalho
commented on August 26, 2024
At first I tried using --install-option="--prefix=" because any args not processed by shiv itself get delegated to pip, so in theory that should've worked! But it didn't seem to make a difference 😞. The other downside to --install-option is that is basically disables the use of wheels which makes installation super slow.
I'm definitely open to an elegant solution to this, but from perusing how this file is handled by distutils and how it's included (by homebrew folks) I'm not hopeful. I'll keep poking at it.
from shiv.
rouge8
commented on August 26, 2024
I think the fix is actually pretty easy -- distutils will also read from a setup.cfg in the current directory, so you can create that and run the pip install from that directory!
This is a minimal diff that seems to work:
diff --git a/src/shiv/cli.py b/src/shiv/cli.py
index d95075d..dbae8e1 100644
--- a/src/shiv/cli.py
+++ b/src/shiv/cli.py
@@ -1,4 +1,5 @@
import importlib_resources # type: ignore
+import os
import shutil
import sys
import uuid
@@ -125,10 +126,13 @@ def main(
interpreter = validate_interpreter(python)
with TemporaryDirectory() as working_path:
+ with Path(working_path, "setup.cfg").open('w') as f:
+ f.write('[install]\nprefix=')
site_packages = Path(working_path, "site-packages")
site_packages.mkdir(parents=True, exist_ok=True)
# install deps into staged site-packages
+ os.chdir(working_path)
pip.install(
python or sys.executable,
["--target", site_packages.as_posix()] + list(pip_args),from shiv.
rouge8
commented on August 26, 2024
Turned that into a PR: #18
from shiv.
lorencarvalho
commented on August 26, 2024
Reopening per conversation in #22 & #18
from shiv.
rouge8
commented on August 26, 2024
I think the reason #18 didn't actually work is pip changes to the directory of the package its installing when it's building from source.
For whoever tries this next: make sure to set PIP_NO_CACHE_DIR=off in your environment when verifying that a fix actually works 😭
from shiv.
rouge8
commented on August 26, 2024
It sounds like what we want is for pypa/pip#4557 to be merged, but in the meantime could probably follow the same approach as Azure/azure-cli#4466
from shiv.
Related Issues (20)
- use the python interpreter (to set the shebang line) when running pip HOT 2
- Accept pipenv's Pipfile.lock in place of pip's requirements.txt HOT 3
- Repetition of `site-packages\build\lib\build\lib\...` HOT 1
- support for packages with a .pth file HOT 3
- Run fails if the built app contains more than 65535 files or is over 2GB HOT 1
- ConfigParser issue with duplicate option entries HOT 2
- Shiv ignores files defined in .gitignore without an option to override and add anyway HOT 2
- Error while finding module specification for '__main__' HOT 2
- How to use single shiv zipapp/pyz for multiple python scripts HOT 1
- Error if output path doesn't exist HOT 1
- The shiv of release page should probably be generated on Windows HOT 1
- Any way to use shiv on read-only filesystem? HOT 2
- Is asyncio supported? HOT 1
- Environments created with symlinked folders (by PDM with cache=True) are not correctly packed
- shiv version 1.0.4 reports the incorrect version of 1.0.3
- Shiv with console script prints the __main__ module dictionary to stderr HOT 1
- Windows: Bug in Interpreter Path Handling
- Is there a way to show a message to the user during the environment bootstrapping phase? HOT 3
- Incorrect bootstrap filename HOT 3
- DuplicateOptionError when there is { in entry_points.txt file
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from shiv.