Comments (10)
the problem has nothing to do with vsc-install, it's true for anything that you try to install and that has an entry in a .pth
in PYTHONPATH.
and the problem might not be easy to fix, but it can be fixed.
is this a system python? out of curiosity, if this is a system python and you would manage to actually build a vsc-base module, would loading the module actually pick the vsc-base from the module via the PYTHONPATH, or also the system one via the .pth entry for vsc-base?
from vsc-install.
using a vsc-base/2.4.2
that was already installed:
$ module load vsc-base/2.4.2
$ ls /apps/gent/SL6/sandybridge/software/vsc-base/2.4.2/lib/python2.6/site-packages/
easy-install.pth site.py site.pyc vsc_base-2.4.2-py2.6.egg
$ cat /apps/gent/SL6/sandybridge/software/vsc-base/2.4.2/lib/python2.6/site-packages/easy-install.pth
import sys; sys.__plen = len(sys.path)
./vsc_base-2.4.2-py2.6.egg
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)
$ python -c "import vsc.utils.generaloption; print vsc.utils.generaloption.__file__"
/usr/lib/python2.6/site-packages/vsc/utils/__init__.py:29: UserWarning: Module vsc was already imported from None, but /vscmnt/gent_vulpix_apps/_apps_gent/SL6/sandybridge/software/vsc-base/2.4.2/lib/python2.6/site-packages/vsc_base-2.4.2-py2.6.egg is being added to sys.path
import pkg_resources
/usr/lib/python2.6/site-packages/vsc/utils/generaloption.pyc
As long as the system vsc-base is backwards compatible, that's not a big issue though (but it does suck).
python -S
may be a way around this, but you can't use this when installing Python packages to force Python to pick up the vsc-install
you want (it breaks setuptools).
from vsc-install.
@stdweird also, I disagree that this is not a vsc-install issue, since we're literally breaking backwards compatibility; we're not longer able to install old vsc-*
packages
and that without a major version bump of vsc-install
...
from vsc-install.
wait, you think that if you load a module and you get something else, this is OK? if EB can't properly install the software and module, why do you install them in the first place?
and if this is OK, it's even easier to fix, just see if you can import the python module you are trying to install, and if you can, print a message like "EB not installing package X, because, well, it doesn't matter anyway and we hope it's backwards compatible".
you can install older vsc- pacakges using an older vsc-install, if it makes you feel better we can increase the major version number, but not sure what it will change.
from vsc-install.
I'm not saying it's OK, I'd like it to b different, but I haven't found a way around it. Suggestions welcome.
from vsc-install.
Actually, we did figure this out for EB, and implemented a fix in the easyblock for EasyBuild, i.e. trash setuptools.pth
in the install prefix because it results in reeling in system Python packages, cfr. easybuilders/easybuild-easyblocks#813.
It's not clear to me why this happens now with vsc-install
too though...
I guess I'll have to dive into the magic that happens in shared_setup.py
.
from vsc-install.
this error message is exactly what prevents the warning about
UserWarning: Module vsc was already imported from None, but /vscmnt/gent_vulpix_apps/_apps_gent/SL6/sandybridge/software/vsc-base/2.4.2/lib/python2.6/site-packages/vsc_base-2.4.2-py2.6.egg is being added to sys.path
We spent all this time on trying to get rid of it, and now you want it back? :p
I do see that you sometimes would want an old vsc-base for reproducing certain things, but then use an old vsc-install to install it, should work.
I do agree that this check breaks backwards compatibility, but the old stuff was doing weird things, that are fixed and are checked for now. So, wont fix.
from vsc-install.
I'm not saying the check shouldn't be there, it's just very annoying that this breaks installing older versions... Also, I obviously don't want the warning back.
Anyway, it doesn't seem to matter too much, since the declare_namespace
stuff is effectively making it impossible to make a particular non-system installation of a vsc-*
package 'win' over the system-wide installed version... So even if we could still install older versions, there's no use, since they won't be used at runtime.
I see no way in which the declare_namespace
in /usr/lib/python2.6/site-packages/vsc/install/__init__.py
can be escaped when using the system Python, which results in that path getting preference; the only way I can find is python -S
, but then other things are broken too...
For now, this is just annoying, it shouldn't break anything, but sooner or later it will, when the system vsc-base breaks backward compatibility. Other than installing old versions of vsc-*
packages that is, because vsc-install
broke backward compatibility already...
W.r.t. EasyBuild, it's sort of fine, since other sites should not be affected (unless they have vsc-install
installed system-wide).
from vsc-install.
well, it only breaks packages that where doing things wrong, new vsc-base won't fail this check, so even if it does something you don't like in future releases, you would still be able to install the current version, right?
from vsc-install.
ok, the issue of having setuptools push system things right in front of the path, right,
python -S will make sure you don't pick up system libraries, you will probably need to install pkt_resources and setuptools in a local path and addt his to your PYTHONPATH if you want vsc-install to keep working, but once you do this you should be isolated from your system environment. (or for easybuild, load the setuptools module)
but always running python -S in easybuild is probably something you want to do anyway?
from vsc-install.
Related Issues (20)
- line too long in doc string
- be able to pass setup.py options through tox
- report setuptools version HOT 1
- vsc-install asks for credentials when using tox HOT 4
- support shellcheck in Jenkinsfile
- vsc-install should test for spaces around operators in 'requires'
- unclosed file HOT 1
- RuntimeError: setuptools >= 41 required to build
- usage of /usr/bin/python in the stripped-env en noenv wrappers HOT 1
- remove flake8 version limit once new prospector that is compatible with pyflakes 2.3 is released HOT 1
- test RPM build via commontest
- run tests with python 3.9
- vsc-install should provide an option to generate github actions workflow file HOT 1
- python 3.12 removes distutils
- allow vsc-install to create correct setup.cfg's HOT 2
- add support for more fields in setup.cfg HOT 1
- check_header function should signal what is exactly wrong
- "python setup.py check -r -s" should be run as a part of "python setup.py test"
- Tests fail: Can't find location of testsuite directory test in /usr/ports/devel/py-vsc-install/work-py27/vsc-install-0.11.2 HOT 4
- "python setup.py bdist_rpm" doesn't replace shebang lines in scripts with recent setuptools HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vsc-install.