Consider using libglvnd from Conan instead of the system's via opengl/system about conan-center-index HOT 9 OPEN

Okay I think I understand your primary concerns, so I'm going to try and summarize them here so you can tell if I understand you or not.

1. The libglvnd package has inadequate testing. Despite being available as a package in CCI, it requires more testing before other recipes are allowed to depend on it. This is due to additional problems that may occur during runtime when loading the GL drivers.

2. The libglvnd library from the host platform or SDK should be preferred over the libglvnd Conan package because the runtime that is installed at the system level is more likely to work with the underlying hardware.

3. Using the libglvnd Conan package over the opengl/system package provides no benefit.

Now that I've summarized these points, I'll add why I'm confused on each one.

1. The libglvnd package has inadequate testing. Despite being available as a package in CCI, it requires more testing before other recipes are allowed to depend on it. This is due to additional problems that may occur during runtime when loading the GL drivers.

On this point, I'm confused why this concern all of sudden on this particular package, as I don't doubt the same concerns exist for OpenCL and Vulkan among others, not that I'm against more testing. There's also the question of whether or not anybody should be using the package if CCI isn't confident enough to use it. Should the package even exist in the first place then or should it have a special disclaimer to let consumers know that using it may cause runtime issues?

2. The libglvnd library from the host platform or SDK should be preferred over the libglvnd Conan package because the runtime that is installed at the system level is more likely to work with the underlying hardware.

I think this makes a lot more sense regarding the actual drivers, like those provided by the Mesa project for Linux. These are very hardware-dependent. libglvnd is an interface to these drivers, that is specifically not hardware-dependent. I still understand the concern regarding system configuration and the loader finding the drivers correctly. I think there is quite a bit of gray area on where this line is drawn between what is too hardware-dependent and what is not. Additionally, I'm quite confused because Vulkan ICD Loader works pretty similarly to libglvnd for looking up device-specific drivers, but we use it as a Conan package instead of a corresponding system package. Why is that?

3. Using the libglvnd Conan package over the opengl/system package provides no benefit.

This question feels wrong. Should every proposed source-based Conan package require justification that it isn't being added as a system package? I think that using source-based packages provides a lot more control and flexibility for consumers. I think the libglvnd Conan package provides all of those benefits in this situation, which is why I think it is still valuable over a system package.

My last primary point of confusion is probably due to the fact that I can't recall any other Conan packages where both a system and a non-system Conan one have existed simultaneously. I'm only aware of a couple recent additions, i.e. the libva and libvdpau packages, which I figured would replace the current system ones.

from conan-center-index.

/cc @jwillikers

from conan-center-index.

@jcar87 You'll need to address the fact that vulkan-loader offers the same loading functionality for Vulkan graphics drivers as libglvnd does for GL drivers. Also, you'll want to address Wayland as well, which is much more important in my scenario than X11.

Also, the fact of the matter is, my company builds graphical applications, but what your saying sounds like go somewhere else if that's the case as that's not something CCI can handle. From your points, we shouldn't have a Qt Conan package that offers graphical components because we can't test them in CCI.

Fyi, vcpkg, HomeBrew, and Spack all offer Mesa packages. Of course, libglvnd is much better to use as a dependency than Mesa directly as it is much simpler and more flexible. I also think it is what Mesa expects to work with anyways.

from conan-center-index.

@jcar87 You'll need to address the fact that vulkan-loader offers the same loading functionality for Vulkan graphics drivers as libglvnd does for GL drivers. Also, you'll want to address Wayland as well, which is much more important in my scenario than X11.

Absolutely! same concerns. But vulkan-loader is vulkan-loader, and OpenGL is OpenGL - the fact that we may have something that is not sufficiently tested or may have issues elsewhere, surely does not justify merging what you propose without due diligence? If anything, if as a result of these proposed changes we are identifying that vulkan also needs looking into, that's a good result: we should look into it.

Also, the fact of the matter is, my company builds graphical applications, but what your saying sounds like go somewhere else if that's the case as that's not something CCI can handle. From your points, we shouldn't have a Qt Conan package that offers graphical components because we can't test them in CCI.

Not at all what I'm saying in the slightest.

When reviewing PRs like with changes like the ones you propose - what I expect is evidence that this approach works, and a justification of what the advatanges are versus what we currently have. Are we solving problems? Are we at risk of introducing new problems? Are we are risk of causing issues to Linux/FreeBSD Conan users that did not exist before?

The fact that this was an issue and somehow looks like without addressing that it would've meant breaking OpenGL if the runtime is provided by Conan - to me indicates that more testing is needed.

The fact that our CI cannot test this is not a problem - we're very happy to do manual testing, document the process and the results in this issue and link it in the relevant PRs, so that there is traceability of why a change was introduced, and what steps were done to validate it where CI couldn't.

, the fact of the matter is, my company builds graphical applications, but what your saying sounds like go somewhere else

Not at all either. In fact, the team has been releasing tools in the Conan client that would help in these cases.
For example, the features presented here https://blog.conan.io/2024/02/20/Conan-2-graph-features.html - could be useful - with a one liner in the profile you could override the use of opengl/system and use libglvnd instead across the entire graph - thus leaving the door open for users with your usecase to follow that path and test it more thoroughly.

Fyi, vcpkg, HomeBrew, and Spack all offer Mesa packages. Of course, libglvnd is much better to use as a dependency than Mesa directly as it is much simpler and more flexible. I also think it is what Mesa expects to work with anyways.

This issue and the PRs are about libglvnd, not mesa? So I'm not sure how that is relevant here.

from conan-center-index.

I'm bringing up Vulkan Loader and Wayland as things that would need to be looked into. I don't think this discussion is just about libglvnd anymore. Mesa is the actual graphics drivers and also the GL implementation when not using libglvnd, so I think it has a lot to do with this discussion.

from conan-center-index.

• Vulkan and Wayland would need opening a new issue with a description of potential problems, and what needs investigating
• For the issue of libglvnd, as with any other proposal in Conan Center - we'll consider it on the basis of the benefits it brings, and whether those benefits are shared by the wider Conan user base. If CI cannot exercise certain code paths, having a sample application that exercises the functionality at runtime would help, as well as identifying potential cases that may be more difficult to test but that may currently work (EGL on Linux tegra, raspberry pi,etc). Will happily test this manually, and leave a proper paper trail of what validation was performed. What I want to avoid is approving these, and then have users report their application have either weird crashes, or no longer work with hardware acceleration.
• We identified in the previous discussion some things that we could already do - there are some things we can advise or do when it comes to cross-building on Linux and the recipe requires xorg or opengl. This could either result in instructing how to produce a Linux sysroot when crossbuilding, or implementing a full-fledged x11 recipe that builds it from source - options are open
• I'd like more insights as to why mesa seems to be OK to have as a separate layer, but not many seem to do this for OpenGL.

from conan-center-index.

* We identified in the previous discussion some things that we could already do - there are some things we can advise or do when it comes to cross-building on Linux and the recipe requires xorg or opengl. This could either result in instructing how to produce a Linux sysroot when crossbuilding, or implementing a full-fledged x11 recipe that builds it from source - options are open

I think one obvious omission to the conversation here is reproducibility. Right now, I have developers using libglvnd and mesa Conan packages to run Wayland just on desktop. This ensures we're using the exact same version of Mesa for development and testing regardless of the Linux distribution, or the specific version thereof.

For cross-compilation, this has been helpful for providing a means to build something against a cross-compiled libglvnd GL implementation, in my case, this is an extension for a Wayland compositor, which can then be deployed on a system with a completely different OpenGL implementation, one that doesn't even use libglvnd but Mesa directly.

* I'd like more insights as to why mesa seems to be OK to have as a separate layer, but not many seem to do this for OpenGL.

Mesa provides the OpenGL, OpenCL, and Vulkan drivers for the underlying hardware. Historically, it also provided the standard OpenGL libraries used to interface with OpenGL. Nowadays, libglvnd is the primary provider of the OpenGL libraries on popular Linux distributions. It was made to work with multiple OpenGL implementations simultaneously, i.e. one from a hardware vendor and another from Mesa. Mesa can be built for use with libglvnd or for use without it as the standalone OpenGL provider. This effectively means that packages can depend on either one for an OpenGL implementation. Of course, libglvnd is newer, which is probably why several other package managers haven't picked it up yet. I imagine that they will. See Homebrew/homebrew-core#77584 (comment) and spack/spack#26301.

from conan-center-index.

I think one obvious omission to the conversation here is reproducibility.

Absolutely - but there's "build time" reproducibility, and then there's the ability to work at runtime - the assumption being that the runtime that is installed at the system level is more likely to work with the underlying hardware, than a runtime provided by Conan that may not work at all (see the hardcoding of distro-specific system paths at compile time).

I think maybe in Conan Center we do not current a model where "please use the system-provided libraries at runtime, but use the Conan-provided libraries/headers at build time" - which would probably satisfy both use cases (similar to the "stub" libraries that CUDA provides). I think this is doable with Conan 2.0, but the recipes in Conan Center currently still support Conan 1.x.

I have developers using libglvnd and mesa Conan packages to run Wayland just on desktop.
Curious to see if this was an issue at all?

For cross-compilation, this has been helpful for providing a means to build something against a cross-compiled libglvnd GL implementation,

Absolutely - but for system-provided libraries this can be handled in a sysroot. I think that's an area of our documentation that may be lacking with a comprehensive example. The sysroot is supposed to help the linker find libraries that are not part of the gcc toolchain, that are expected in the target system. Obviously, if all the libraries you need are Conan packages, Conan removes the need to maintain a separate sysroot - but my experience is that except for the simplest cases, you will need one anyway, for anything that strictly needs a system library. A good example: cross-building CUDA code for the NVIDIA Jetson - you can't mix and match (or at least not until recent version of jetpack) versions of CUDA libraries and the OS side (unlike what you can do on PCs).

How to properly build a sysroot is something has some historical caveats - see crosstool-ng. On Apple the system layer is fully provided by the SDK, including a sysroot. The same goes for Android and QNX, but Linux is more is different. My point being that is not a strict requirement for all libraries to be provided by Conan if the system-only libraries are already included in the sysroot (or if you are building on a multilib distro and targetting the same distro version) this is something we need to document better.

This effectively means that packages can depend on either one for an OpenGL implementation. Of course, libglvnd is newer, which is probably why several other package managers haven't picked it up yet. I imagine that they will. See Homebrew/homebrew-core#77584 (comment) and spack/spack#26301.

Looks like libglvnd predates vpckg, and the discussions you have linked actually reinforce the idea of following a more cautious approach: the homebrew discussion seems to indicate that the entire stack would need to be tested (all libglvnd or ALL mesa), and from the spack discussions, seems like they tried it, reverted it, and haven't addressed since they floated the idea in 2020 - that seems to be dilated in time.

As I said, I think there's reason to be cautious here, and identify the issues that are all other package managers have been avoiding. I think it's a good idea on paper since it's the newer, I just want to make sure that we dont' leave anyone behind.

from conan-center-index.

I fully agree that, if using libglvnd instead of opengl/system works, it provides all the benefits you've mentioned. I have never said that it provides no benefit.

As a maintainer of this repository, I want to ensure that changes are sufficiently vetted, and that our approval of a pull request gives us confidence that it will work as intended. With regards to OpenGL, I feel like Conan Center follows pretty much the same approach as similar repositories (pypi, homebrew, vcpkg, spack) and I can see signs that they have hesitated or not fully made the transition yet due to technical concerns. Now, given that the approach we currently use is known to work (despite some limitations regarding cross-building and hermetic builds), and that the maintainers of the other repositories have hesitated to make this very exact change, I don't think it is unreasonable for us to be concerned and to clarify and prevent our users experiencing any issues.

I want to help you move this forward, because I agree with the benefits that you mention, so here are my concerns from a technical standpoint:

• merging #23439 without merging this first: #23500 - would have resulted in an application using glfw not picking up the OpenGL implementation in the system due to not looking in the right place, correct? If that is the case, I think that's a good example of PR reviews working before the users experience issues.
• is it certain that the changes in #23500 will work for most Linux distros? Is it always /usr/share/glvnd/egl_vendor.d ? Do we have knowledge of any distro that places it somewhere else?
• Even if those cases where that is indeed the directory, who places files there? Are users likely to find that their distro already has drivers in there?
• You mention in #23500 that you are not sure if that path is the one for FreeBSD, yet all the other PRs explicitly switch to use it on FreeBSD as well. This kinda reads as "this maybe works in FreeBSD" - can we have more confidence? Otherwise I think it may be best to leave FreeBSD as it currently is, and FreeBSD users can provide feedback and request this in the future.
• In the homebrew link you provided, they are hesitant to move to libglvnd unless they move "the whole stack". Does this mean that all PRs that switch to use libglvnd need to be merged at once/closely together to avoid any issues, or can they be merged as they come? If they cannot be merged at once, and we have a dependency graph that has both libglvnd and opengl/system, what are the potential issues? I want to avoid what happened with the libuuid update.
• The proposed changes for spack in the link you provided, document the usage this way: https://github.com/spack/spack/blob/43235601313f003d3e7d757cc9b040504e70001b/lib/spack/docs/getting_started.rst#L1025-L1040 . It looks like consumer needs to tell the package manager where to find the driver, and which driver to use, in what appears to be the equivalent of a Conan conf such that an environment variable is used. IF we move forward with the changes you are proposing, do Conan users now need to have a setup similar to that? And if the answer is no, it is a case of "just works", or do the users need to have something else installed on their systems, completely independent of a Conan package (system or not).
• Some recipes (glew and pcl as far as I can see) already seem to depend on opengl/system (which brings the system libglvnd), but also the Conan provided mesa-glu/9.0.3, which brings the Conan provided libglvnd by default on Linux, as a transitive dependency. Is this not a potential conflict? Which one gets picked up at build time, and which one gets picked up at runtime? Are these not incompatible? Is it possible that because they have a pure C ABI and the API is pretty much set in stone, that perhaps there is a mix up that is not causing any issues (yet)?
• Is the libglvnd recipe a direct replacement of opengl/system for all intents and purpose? that is, compile time (headers), link time (static linker), runtime (dynamic linker). if the answer is yes to all, should these be a "conflict" ?
• where does egl/system play into this? it also installs the systems' libglvnd. Does libglvnd not provide the EGL libraries as well?
• Could there be other approaches considered, such as having a unified opengl recipe (whichever version/variant) acting as a selector for what actually happens under the hood? i.e., one could use the system's, or one could use libglvdn - moving the choice to the consumer?
• Is it not possible to use [replace_requires] to redirect opengl/system to libglvnd/[version], for those users that require it, without doing anything else? The feature was specifically designed to accommodate users who may have other needs than the Conan Center recipe defaults.
• the spack thread has this question, which I quote verbatim, as it applies to our users as well: "would some users potentially not be able to use this on older systems?"

• This discussion also seems relevant - spack/spack#26301 (comment) - in particular this quote ". So we need to have a way to enable an externally provided opengl with a spack-provided libglvnd. " - do we have similar concerns? That is, an 'externally' provided implementation using a Conan-provided libglvnd. Looks like their package design what much informed by this constraint.

from conan-center-index.