pin_run_as_build vs run_exports for new packages about conda-forge-pinning-feedstock HOT 14 OPEN

This was absolutely the plan. The poll results were:

9 votes total:

Since the language was changed a little way into the poll, there are 8 votes for run_exports, and 1 against.

from conda-forge-pinning-feedstock.

It's not "I don't like the result." It is "I don't like how the poll was conducted." Many of us at least once were if not are currently scientists. My concern is the procedure used here was poor and for the reasons already cited.

from conda-forge-pinning-feedstock.

Sorry Mike, but that wording is even less clear than the original poll.

It doesn't even use run_exports in there. Also don't follow what the first option even means or why there are 3 when we seem to want only 2. The fact that only 9 people voted on it (unlike our Windows 32 poll, which got 91 responses) demonstrates people have no idea what is being asked. Mixing the poll with different wording is sure to cause bias in our already poor sample size.

We should redo the poll with clear wording (as I'd mentioned on Gitter chat after it was released).

from conda-forge-pinning-feedstock.

The full survey text is below. The screenshot was only the text of the choices. I did edit it according to your input at the time. There are 3 options because I edited the choices after your input. The second sentence in the first 2 choices was added. No one had voted for the "Packages serve multiple roles..." option at the time that I added the extra language, so there's 3 results, not 4 as you might expect.

9 responses is actually what I would expect. This is a very low-level issue that most people just don't care about. If you still demand a re-vote, you should organize it. This vote was and is valid until you do so.

This poll is a stylistic choice for the Anaconda and conda-forge ecosystem, started in #58. The ultimate functional outcome of either choice is roughly equivalent. This choice affects both Anaconda and Conda-forge because of the effort to unify recipes.

TL;DR:

1. Should packages themselves communicate how they are used (in terms of setting ABI expectations)? Packages would serve more limited roles (a -devel package at build time, and a lib- at runtime). Most packages would not be split - only those that currently serve multiple roles within the build/host phase, or those that would benefit from splitting, in terms of saving space or speeding up build by reducing duplicate build.

2. Should downstream recipe authors decide how they want a package to behave for their particular recipe? Packages may serve multiple roles.

Moar details, plz:

1. Add run_exports metadata to packages that are expected to be linked against. This imposes runtime compatibility bounds. When these dependencies are present in the host section, they will add their exported dependencies to the run section, and the run section need not explicitly name those run dependencies. run_exports can be different on a per-version basis, as they are part of the recipe itself. Packages that are header-only (some of boost), or are used in ways that do not link against shared libraries (numpy as a python library) would need to be split off into their own separate packages. Another way to say this is "if there's more than one behavior with the package in the host section, we should disambiguate it." For example, see Anaconda's numpy recipe: https://github.com/AnacondaRecipes/numpy-feedstock/blob/master/recipe/meta.yaml. These package splits may save space in some cases, but with boost and numpy, there is little gain. Metapackages will be used to maintain backwards functionality for the older package names. This should not be interpreted to mean that everything should be split up - only where it clarifies how a package is used, or where there are significant size or build time advantages.

2. Use pin_run_as_build. This only takes effect when a given dependency name is present in both host and run sections. It is decoupled from the recipe, so it can't operate differently for a given package on a per-version basis. If a package is present only in build or only in host, no automatic constraints are applied. This maintains the status quo with packages as they are today, with regards to not needing splitting work.

from conda-forge-pinning-feedstock.

Right, I recall this text and the concern before was this was quite lengthy and missed some important things. There is a rather lengthy discussion about this starting here. Expect this is a large part of why there are only 9 participants.

I'm not sure that I would justify a change of this magnitude by only 9 participants. There are almost twice as many (committing) contributors just to conda-forge-pinning. Not to mention the numerous people discussing pinnings at this repo. So I think there are a lot more people interested in this subject and the poll did not capture them, which I still find concerning.

As stated before, I don't think a poll that changes its wording part way through should be considered valid. Wording has a huge impact on poll outcome. Doing it midway through in my mind already makes it invalid.

Independently there is a discussion of what the community does next, but my main contention right now is this was not conducted properly and so is invalid.

from conda-forge-pinning-feedstock.

I'm not sure that I would justify a change of this magnitude by only 9 participants. There are almost twice as many (committing) contributors just to conda-forge-pinning.

We cannot force people to vote and the usual active core members at certain time is less than 9. I'd say that is pretty good and I don't see a reason to keep halting this over what seems like "I don't like the result."

from conda-forge-pinning-feedstock.

It's not "I don't like the result." It is "I don't like how the poll was conducted."

Really? Simple Google form with plenty of time to read and @msarahan at your disposal to answer any questions. What else do you want? Does everything need to be to your liking to pass on conda-forge? Please re-consider what you are doing here, it is not healthy!

from conda-forge-pinning-feedstock.

Start a new poll, John. Show us what a properly conducted poll looks like. I didn't restart the poll because I didn't want to waste people's time. People had the power to change their votes.

I find this experience very insulting to the effort that I put into it, and I don't appreciate it.

from conda-forge-pinning-feedstock.

Start a new poll, John. Show us what a properly conducted poll looks like.

I really prefer if we don't go that route. We had 9 out of 12 core members voting (I did not count new ones added after the pool was conducted). Usually in our meetings we get 4-5 active members making 9 is a pretty good outcome. I don't see the result changing if we conduct a new poll just b/c 1 does not like the text on the poll.

Please consider the effort and time people put into this and respect the result.

from conda-forge-pinning-feedstock.

I would like to state one hypothetical scenario and would like to know how it would be handled in pin and pin_run_as_build world only.

Let's say there is a library libSuper 2.3 and there is a Conda package for it. Let's say it has no run_exports and the pin is on: 2.3 and pin_run_as_build is: x.x . Now, assume that majority of other packages using this package as a dependency are happy with this pinning. Later, a new version of libSuper comes out, 3.0 (again, no run_exports), and provides additional features. Now, if another Conda package relies on this new functionality and wants libSuper >=3.0, how would it go about using it? Would the existing pinning in pin_run_as_build create some conflict?

from conda-forge-pinning-feedstock.

In either world, the outcome is the same. The package depending on libSuper will need to be rebuilt. In both the run_exports world and the pin_run_as_build world, which version of libSuper is present at build time is determined by pin. An update to conda-forge-pinning would be necessary in either case, and in either case the constraint on the final package would be similar.

The only difference here is who gets to say what the compatibility bounds are. With pin_run_as_build, it's the people who control conda-forge-pinning. There is no per-version bound. With run_exports, it's the people who control libSuper. The bound is associated with a specific revision of the recipe.

from conda-forge-pinning-feedstock.

Okay, assuming there is no pin_run_as_build but only run_exports and pin:

In case both library versions have to be maintained (assuming libSuper 3.0 makes a breaking change), and the fancy Conda package depending on libSuper can't be built with the older one (but there are other packages which still require the 2.x version of the library and also keep getting updated), what would be the recommended way to avoid having it built with the older value in pin? Use a selector on skip? Explicitly specify the version constraint in requirements/host and hope that CB3 will do the right thing?

I ask about this because CF relies heavily on automation. One cannot just hand edit the pinning file and then later revert it.

from conda-forge-pinning-feedstock.

I'm not clear on what you mean. I understand you're talking about the annoying state we have sometimes where a new low-level package has come out, and not everyone supports it yet.

This is a weakness of the global pinning approach in general. There is no single correct value for pin. The best we can do is:

• set pin for the most common case
• hard-code values in requirements/host otherwise

Now those hard-coded values really aren't any different from pin as far as compatibility bounds are concerned. They both set the value that CB3 uses as the basis for any pin_run_as_build, and obviously you'd only get run_exports in action if the package in host actually has run_exports.

I don't see the connection to automation, though. How could we improve automation here? How does pin_run_as_build vs. run_exports affect automation? If the compatibility bounds change between libSuper 2 and 3, then the global pinning of pin_run_as_build is not desirable. It will be either too tight or too loose, unless 2 and 3 behave similarly.

from conda-forge-pinning-feedstock.

Apologize for the comment on automation, I was thinking about something different. I may have gone off on a tangent there.

from conda-forge-pinning-feedstock.