Interest in manylinux wheels about btrees HOT 7 CLOSED

I think it is a very good idea!

from btrees.

I notice that those require sudo: true. It's documented that sudo: true builds run on a different infrastructure (non container-based) which is substantially slower to boot (up to 52 seconds vs up to 6), and anecdotally, I seem to recall that the queue to even be able to start sudo: true builds tended to be substantially longer than the container-based environments (whether that's still true or not now, I can't say; maybe someone has more recent experience). All of which boils down to taking much longer to get feedback from the CI on a push/PR to those repositories. And since the builds for an organization (zopefoundation) are effectively queued together, that potentially slows down feedback for other repositories too.

Adding the Mac builds already made it frequently take substantially longer (hours sometimes) to get Travis to "go green" for those projects...but that wasn't a huge concern because the Linux builds that covered all the different supported Python versions and PURE_PYTHON variants, etc, typically finished within a minute anyway, so an experienced contributor was already able to get the most useful feedback quickly. (And could then cancel the pending Mac builds to be able to start the next commit build in case of failure.) And I can see the benefit to Mac users for having pre-built wheels, so it seems worth the minor hassle.

But the benefit to Linux users looks less enticing to me, since Linux users most often already have compilers installed (or if they're doing production deployments have already solved the problem of not having compilers on web server machines, etc, by using their own egg caches/devpi/docker images/whatever). There are no complicated external dependencies to install for these libraries either, and downstream packagers don't rip them apart to undo bundled dependencies (as was the rationale for gevent). In some cases, we'd conceivably be downgrading the quality of the code that Linux users would be running, since manylinux1 wheels are built with a very old compiler.

All of which makes me roughly -0 on this, I guess: not a fan, but not opposed in the face of good arguments.

If we do this, though, I would suggest we turn on a few Travis settings for those repos to limit the queue damage: first, the new setting that makes it cancel old builds when new commits to a branch come in (good for making sure that only the most recent commit builds, typically after a quick bugfix), and potentially the setting that only PRs go to Travis, not branches (cuts the number of builds for a PR in half, and since the workflow is to never commit to master and rarely to do multiple pushes to a branch before opening a PR, probably not changing the feedback most of the time).

from btrees.

Here are measures that can taken to mitigate delays for zopefoundation builds

1. sudo isn't actually necessary anymore
2. we can use allow_failures on the manylinux builds along with fast_finish so that, per build, providing feedback developers is no slower. This would have the effect of turning the build green as soon as the current builds for linux and mac os x are done. However, the manylinux builds would still be run, which means more builds in zopefoundation queue
3. We quickly bail out of the manylinux build except on tagged commits. Since we only want manylinux builds in order to make manylinux wheels, this could work well. If we felt confident about mac os x, we could do the same for those builds. This would both mean a minimal increase in per-build feedback (for not tagged commits), but also a minimal contribution to queue congestion.

from btrees.

sudo isn't actually necessary anymore

That'd be cool! But as far as I can tell from the docs using Docker still requires sudo. And building manylinux requires docker. Can you let me know where I'm wrong, please?

from btrees.

You are right. Sorry.

from btrees.

@mgedmin @jamadden I made a PR with option 3. #67

from btrees.

done for this repo.

from btrees.