Ensure that we always get the dev versions of the relevant packages about integration-test HOT 7 OPEN

I think the easiest way to make sure this works correctly is to spin up our own PyPI endpoint locally.

We could also, for each test, put the artefacts we want to test (latest setuptools wheel, pip master sdist, etc) in a directory packages and use pip options --no-index --find-links=./packages.

from integration-test.

We could also, for each test, put the artefacts we want to test (latest setuptools wheel, pip master sdist, etc) in a directory packages and use pip options --no-index --find-links=./packages.

Can we guarantee that virtualenv is using these pip options, though? If we don't give the integration tests access to the public PyPI during the installation tests, we're guaranteed that the tests were run using the dev versions. If not, we may install setuptools==41.0.0.deva394ad33cffeda in our current environment only to have pip and virtualenv install setuptools==40.7.0 in their respective isolated environments.

from integration-test.

Can we guarantee that virtualenv is using these pip options, though?

According to https://github.com/pypa/virtualenv/blob/bcce79d2e827c32153a1a199ddbb99f289f7694d/virtualenv.py#L1014 it seems we cannot ^^

So a small http.server serving our artefacts with the matching PIP_INDEX_URL might be simpler.

from integration-test.

The nice thing about us owning all these packages we can make the necessary changes to allow this.

from integration-test.

I'm not 100% sure what tests we're actually planning on doing, but we could build virtualenvs by using --no-setuptools and doing all of our installs "by hand". Or by putting our dev builds in a virtualenv_support directory (which is where virtualenv picks them up from).

Ideally, a local index is the best option, assuming no tools clear environment variables (I know tox does for some, but I don't imagine it would be the pip ones, I think it's more PYTHONPATH and the like).

from integration-test.

Ideally, a local index is the best option, assuming no tools clear environment variables (I know tox does for some, but I don't imagine it would be the pip ones, I think it's more PYTHONPATH and the like).

If we go with a local index supplying all the files we need, I think we would want to do it in two stages - first stage builds the index, then the second stage uses it. I think we can probably mess with /etc/hosts or do something equivalent to prevent access to public PyPI at the network level in stage 2, then we don't have to worry as much about what respects environment vars.

from integration-test.

I’ve had very good experiences with devpi. Although it’s not precisely PyPI, it has some nice properties (lightweight, pure-python, supports overwriting releases) and simulates PyPI pretty well. I’d recommend it.

from integration-test.