Comments (15)
GrahamDumpleton
commented on July 21, 2024
Confirmed that this is fixed by fixing #57.
To test this, one can run:
sti build https://github.com/GrahamDumpleton/openshift-sti-python-sample.git openshift/python-27-centos7 python-sample-app
The repo being pointed at has a requirements.txt that installs two special modules for test. When run it yields:
W0828 16:30:11.868852 04863 docker.go:299] BuilderImage uses deprecated environment variable STI_SCRIPTS_URL, please migrate it to io.openshift.s2i.scripts-url label instead!
W0828 16:30:12.143298 04863 sti.go:127] Clean build will be performed because of error saving previous build artifacts
I0828 16:30:12.536852 04863 sti.go:393] ---> Copying application source ...
I0828 16:30:12.544565 04863 sti.go:393] ---> Installing dependencies ...
I0828 16:30:13.016062 04863 sti.go:393] Downloading/unpacking https://dl.dropboxusercontent.com/u/22571016/module1-1.0.tar.gz (from -r requirements.txt (line 1))
I0828 16:30:14.970469 04863 sti.go:393] Downloading module1-1.0.tar.gz
I0828 16:30:14.984304 04863 sti.go:393] Running setup.py (path:/tmp/pip-tMGbPB-build/setup.py) egg_info for package from https://dl.dropboxusercontent.com/u/22571016/module1-1.0.tar.gz
I0828 16:30:15.105179 04863 sti.go:393]
I0828 16:30:15.122645 04863 sti.go:393] Downloading/unpacking https://dl.dropboxusercontent.com/u/22571016/module2-1.0.tar.gz (from -r requirements.txt (line 2))
I0828 16:30:15.528597 04863 sti.go:393] Downloading module2-1.0.tar.gz
I0828 16:30:15.534032 04863 sti.go:393] Running setup.py (path:/tmp/pip-ZlSNRU-build/setup.py) egg_info for package from https://dl.dropboxusercontent.com/u/22571016/module2-1.0.tar.gz
I0828 16:30:15.636441 04863 sti.go:393] sh: module1: command not found
I0828 16:30:15.649359 04863 sti.go:393]
I0828 16:30:15.668118 04863 sti.go:393] Installing collected packages: module1, module2
I0828 16:30:15.669264 04863 sti.go:393] Running setup.py install for module1
I0828 16:30:15.793085 04863 sti.go:393]
I0828 16:30:15.821481 04863 sti.go:393] Installing module1 script to /opt/app-root/src/.local/bin
I0828 16:30:15.828454 04863 sti.go:393] Running setup.py install for module2
I0828 16:30:15.933136 04863 sti.go:393] sh: module1: command not found
I0828 16:30:15.958622 04863 sti.go:393]
I0828 16:30:15.980009 04863 sti.go:393] Installing module2 script to /opt/app-root/src/.local/bin
I0828 16:30:15.994616 04863 sti.go:393] Successfully installed module1 module2
I0828 16:30:15.995165 04863 sti.go:393] Cleaning up...
There are two instances of:
sh: module1: command not found
The first is okay as that is when python setup.py egg_info is run on each module and so it will not exist. It should though succeed in the second situation though, which is after module1 should have been installed.
If the fix in #57 is made, then as expected one gets:
E0828 16:33:58.605732 04951 sti.go:419] exec: "/usr/local/sti/save-artifacts": stat /usr/local/sti/save-artifacts: no such file or directory
W0828 16:33:58.673400 04951 sti.go:127] Clean build will be performed because of error saving previous build artifacts
I0828 16:33:59.069042 04951 sti.go:393] ---> Copying application source ...
I0828 16:33:59.069466 04951 sti.go:393] ---> Installing dependencies ...
I0828 16:33:59.447724 04951 sti.go:393] Downloading/unpacking https://dl.dropboxusercontent.com/u/22571016/module1-1.0.tar.gz (from -r requirements.txt (line 1))
I0828 16:34:00.753715 04951 sti.go:393] Downloading module1-1.0.tar.gz
I0828 16:34:00.766193 04951 sti.go:393] Running setup.py (path:/tmp/pip-m1Ikun-build/setup.py) egg_info for package from https://dl.dropboxusercontent.com/u/22571016/module1-1.0.tar.gz
I0828 16:34:00.873978 04951 sti.go:393]
I0828 16:34:00.889913 04951 sti.go:393] Downloading/unpacking https://dl.dropboxusercontent.com/u/22571016/module2-1.0.tar.gz (from -r requirements.txt (line 2))
I0828 16:34:01.355267 04951 sti.go:393] Downloading module2-1.0.tar.gz
I0828 16:34:01.360139 04951 sti.go:393] Running setup.py (path:/tmp/pip-IhxyY9-build/setup.py) egg_info for package from https://dl.dropboxusercontent.com/u/22571016/module2-1.0.tar.gz
I0828 16:34:01.453760 04951 sti.go:393] sh: module1: command not found
I0828 16:34:01.465874 04951 sti.go:393]
I0828 16:34:01.482197 04951 sti.go:393] Installing collected packages: module1, module2
I0828 16:34:01.482494 04951 sti.go:393] Running setup.py install for module1
I0828 16:34:01.594591 04951 sti.go:393]
I0828 16:34:01.615238 04951 sti.go:393] Installing module1 script to /opt/app-root/src/.local/bin
I0828 16:34:01.630538 04951 sti.go:393] Running setup.py install for module2
I0828 16:34:01.767281 04951 sti.go:393] module1
I0828 16:34:01.767814 04951 sti.go:393] module1
I0828 16:34:01.768305 04951 sti.go:393] module1
I0828 16:34:01.768796 04951 sti.go:393] module1
I0828 16:34:01.769313 04951 sti.go:393] module1
I0828 16:34:01.769824 04951 sti.go:393] module1
I0828 16:34:01.770383 04951 sti.go:393] module1
I0828 16:34:01.770602 04951 sti.go:393] module1
I0828 16:34:01.774093 04951 sti.go:393] module1
I0828 16:34:01.774116 04951 sti.go:393] module1
I0828 16:34:01.774124 04951 sti.go:393] module1
I0828 16:34:01.774129 04951 sti.go:393] module1
I0828 16:34:01.774135 04951 sti.go:393] module1
I0828 16:34:01.774140 04951 sti.go:393] module1
I0828 16:34:01.797786 04951 sti.go:393]
I0828 16:34:01.817638 04951 sti.go:393] Installing module2 script to /opt/app-root/src/.local/bin
I0828 16:34:01.831551 04951 sti.go:393] Successfully installed module1 module2
I0828 16:34:01.832118 04951 sti.go:393] Cleaning up...
E0828 16:34:01.859578 04951 sti.go:419] chmod: chmod: cannot access '/opt/app-root/src/.pki/nssdb'cannot access '/opt/app-root/src/.pki/nssdb': Permission denied: Permission denied
E0828 16:34:01.860623 04951 sti.go:419]
In second case module1 is found and run okay.
FWIW. Using a per user Python install area isn't necessarily the best idea. Some modules may not cope properly with that when being installed if they have to do special stuff. It may have been better to use a proper Python virtual environment.
from s2i-python-container.
rhcarvalho
commented on July 21, 2024
It may have been better to use a proper Python virtual environment.
👍 exactly what I've been saying in the past...
from s2i-python-container.
GrahamDumpleton
commented on July 21, 2024
With Docker based images there is no need to be relocating the Python virtual environment like under OpenShift 2, so one will not have the problems which attempting to do that caused. I can't therefore see a good reason not to use a proper Python virtual environment now.
In the meantime I need to go back and work out why one of my packages will not install properly under a per user Python installation area.
from s2i-python-container.
bparees
commented on July 21, 2024
@sYnfo are you still intending to work on this?
from s2i-python-container.
bparees
commented on July 21, 2024
@GrahamDumpleton since #57 is fixed, is this fixed, per your comment above?
from s2i-python-container.
GrahamDumpleton
commented on July 21, 2024
I presume this still isn't released in builder images actually in use by OpenShift.
When trying on our new dedicated cluster I still see the original problem I had. It has:
$ oc describe is python -n openshift
Name: python
Created: 9 days ago
Labels: <none>
Annotations: openshift.io/image.dockerRepositoryCheck=2016-02-02T18:41:23Z
Docker Pull Spec: 172.30.145.153:5000/openshift/python
Tag Spec Created PullSpec Image
2.7 registry.access.redhat.com/rhscl/python-27-rhel7:latest 9 days ago registry.access.redhat.com/rhscl/python-27-rhel7:latest 93d5039ac0fd9c1a9361b3459fdb005cddbdab694afe8d09abf18c064abebf20
3.3 registry.access.redhat.com/openshift3/python-33-rhel7:latest 9 days ago registry.access.redhat.com/openshift3/python-33-rhel7:latest 610e9de804253de6e00dc3e5d32407cf0840a9eb9ddd29d072f032e720121386
3.4 registry.access.redhat.com/rhscl/python-34-rhel7:latest 9 days ago registry.access.redhat.com/rhscl/python-34-rhel7:latest bae1743ada780f4f14ba992fb5719ecd8cb2360480546280369050e597f98b3f
latest 3.4 9 days ago registry.access.redhat.com/rhscl/python-34-rhel7:latest bae1743ada780f4f14ba992fb5719ecd8cb2360480546280369050e597f98b3f
It would have used Python 3.4 variant as latest.
I get the same problem with it.
If however I create my own builder using:
oc new-build https://github.com/openshift/sti-python.git --context-dir=2.7 --name=python27-centos7
and then use it with my test application which triggers the problems then it is okay.
So if it is the case that RHEL Python 2.7 builder is still old, then changes in repo which would be released later do appear to address the issue at least.
from s2i-python-container.
bparees
commented on July 21, 2024
The path in the current rhel python 2.7 looks like what you said it should be in #57:
$ docker inspect registry.access.redhat.com/rhscl/python-27-rhel7:latest | grep PATH
"PATH=/opt/app-root/src/.local/bin/:/opt/app-root/src/bin:/opt/app-root/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
and my imageid matches what you're reporting from your cluster.
So is there a different issue here perhaps?
from s2i-python-container.
GrahamDumpleton
commented on July 21, 2024
My test would have used Python 3.4 image, not 2.7, as I just let it grab latest.
Let me check with 2.7.
from s2i-python-container.
bparees
commented on July 21, 2024
3.4 looks right too:
$ docker images | grep registry.access.redhat.com/rhscl/python-34-rhel7
registry.access.redhat.com/rhscl/python-34-rhel7 latest bae1743ada78 10 weeks ago 462 MB
$ docker inspect registry.access.redhat.com/rhscl/python-34-rhel7 | grep PATH
"PATH=/opt/app-root/src/.local/bin/:/opt/app-root/src/bin:/opt/app-root/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
from s2i-python-container.
GrahamDumpleton
commented on July 21, 2024
Testing with 2.7 fails for me in the same way with my test.
I can't get into a running container and manually do test because the RHEL images are still missing the libbss_wrapper stuff such that generate_container_user works and so it isn't possible to run pip after getting in using oc rsh.
The latest master may well have worked for me, because the test would actually have fallen through to using the apxs program installed as part of httpd-devel in that image if it wasn't finding override version called mod_wsgi-apxs that my mod_wsgi-httpd package installed. I will need to go back and check again when using image created from master.
Because this whole issue only arose for me when I was trying to work around absence of httpd and http-devel packages, then likely not a big drama if there is some reason that PATH not picked up correctly across packages being installed using pip. The updated PATH will still be of use in normal user case where they change the working directory, and PATH does show when using oc rsh that is no longer a relative path.
I will though keep digging just a little longer to try and understand why my test still fails. I was sure that I had tested it okay some time back when I had forked the builder repo and created my own version with the change in it.
from s2i-python-container.
GrahamDumpleton
commented on July 21, 2024
After some more checking, although the PATH change was still necessary, I may be now hitting a subsequent issue. That is the fact that a quite out of date pip version is being used which behaves differently to more recent pip versions. The pip version installed in Python 2.7 image is 'pip-1.5.6'. That version is coming up to 2 years old in May. There has been 21 releases of pip since then.
If I update the pip version within the running container to the latest version (8.0.2) and then do my tests then it works okay.
Old pip versions will get a similar reaction as old Python versions. Users wants the latest, especially since newer versions support wheels, which can be used to speed up installation of packages.
from s2i-python-container.
rhcarvalho
commented on July 21, 2024
On the pip version side, I remember when we had only the Python 3.3 image the pip version was much older than 1.5.6, even +1 year older than that. We ran into a bug that was fixed from 1.5.6 onwards, and 1.5.6 was what we had packaged in RHEL, but not in SCL.
In the Py3.3-centos image we still install that version using easy_install instead of yum:
3.3/Dockerfile#L36
I noticed that discrepant line just recently when I was adding httpd to the Dockerfiles, and verified that all image versions are shipping with pip 1.5.6. Indeed pretty old.
The question is are we going to get newer versions into centos/rhel packages, or would we allow ourselves to easy_install or pip install -U a newer version?
from s2i-python-container.
rhcarvalho
commented on July 21, 2024
@hhorak @sYnfo one more update request, would it be possible to get a newer pip in all python images?
The Python 2.7, 3.3 and 3.4 images we have now all have pip 1.5.6.
from s2i-python-container.
hhorak
commented on July 21, 2024
@rhcarvalho We cannot use anything else than what is in the SCL, so in case you need something newer, please, report a BZ for appropriate collection with the reasoning (please, also mind that we do not update all the collections for every release): https://bugzilla.redhat.com/enter_bug.cgi?product=Red%20Hat%20Software%20Collections
from s2i-python-container.
pkubatrh
commented on July 21, 2024
Closing via #159
from s2i-python-container.
Related Issues (20)
- No Action Required !! Testing automation workflow HOT 1
- No Action Required !! Testing automation workflow HOT 4
- Remove verification of installed packages HOT 3
- 3.9 Readme Instructions unclear HOT 6
- Python 3.10 RHEL image missing in RHEL Container catalog? HOT 4
- Unable to build a Python image from scratch on MacOS HOT 1
- Python 3.9+ for Centos 7 docker images HOT 4
- tests: test case should fail early when the prepare function fails HOT 3
- Distgen errors HOT 11
- Incorrect py-3 image HOT 11
- Support gunicorn >=20.1.0 defaults (do not require APP_ environment variables) HOT 1
- Add RHEL images for Python 3.11 HOT 4
- ERROR: No matching distribution found for numpy==1.19.2 HOT 4
- Use PIP_INDEX_URL with pipenv HOT 1
- ubi9/python-311:latest is using python3.9-rpm HOT 2
- Publish arm64 images HOT 8
- rh-python38 failed on `'npm-virtualenv-uwsgi-test-app' run_s2i_build` & `'pin-pipenv-version-test-app' run_s2i_build` HOT 2
- python311-devel not found by microdnf in python 3.11 minimal EL8 and C9s variants HOT 1
- Documented pull example quay.io/sclorg/python-39-minimal isn't a valid URL HOT 5
- Cannot resolve internal host HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from s2i-python-container.