Need to update the base of CUDA docker images to CentOS 7 about docker-images HOT 18 CLOSED

Just a note here, we are currently waiting for CentOS 7 CUDA 11 images to be published. More details on the current status in this comment.

from docker-images.

CF packages work just fine on CentOS 7 even though built in CentOS 6.

from docker-images.

We set up two variants of docker images for CUDA 10.1 & 10.2 that is based on CentOS 7, and put them in production in conjunction with existing ones.

Sure, but you'll also need to add __glibc >=2.17 to all the recipes using CentOS 7 to make sure that users using older GLIBC will not get these new packages that don't work on their systems.

from docker-images.

cc: @jakirkham

from docker-images.

FYI, according to the Release Notes of CUDA 10.2, RHEL 6.x will be dropped in the upcoming CUDA 11:

Note that support for RHEL 6.x is deprecated and support will be dropped in the next release of CUDA.

from docker-images.

Where is cuTENSOR coming from?

We've had this discussion before and updating the base image just to accommodate a proprietary binary blob is no. There are still many people using CentOS 6.

from docker-images.

@isuruf

Where is cuTENSOR coming from?

Like cuDNN, cuTENSOR is available from NVIDIA Developer Program. @jjhelmus built the former in Anaconda's channel locally and then upload the package, and I verified that the same approach applies to cuTENSOR.

We've had this discussion before and updating the base image just to accommodate a binary blob is no. There are still many people using CentOS 6.

Unfortunately we don't get to choose when CUDA 11 is out (see my above #137 (comment)). The base won't be able to support it.

from docker-images.

One more thing: as of cuTENSOR 1.0.1 only CUDA 10.1 and 10.2 are supported, so a compromise could be that we only update the base for 10.1+

from docker-images.

One final note 😅: I am not saying this change is urgent/critical (to CuPy or any other CUDA-based packages). I'm simply bringing this issue to CF core devs' attention. This doesn't have to be addressed right now, but an update plan should be considered/revisited in the near future.

from docker-images.

Why do you want to update the whole conda-forge system to CentOS 7 just to accommodate a proprietary software? Why not just change the few feedstocks you need it for CentOS 7?

from docker-images.

Why not just change the few feedstocks you need it for CentOS 7?

@isuruf Could you elaborate on how to do this please?

from docker-images.

You can override docker_image variable in recipe/conda_build_config.yaml.

from docker-images.

But how do I know the package would be compatible with the rest of CF packages that also depend on cudatoolkit and other CUDA packages?

from docker-images.

Thanks for confirmation, @isuruf. It's good to know.

from docker-images.

Sorry @isuruf, one follow-up question:

CF packages work just fine on CentOS 7 even though built in CentOS 6.

Were you implying the converse wouldn't work? (Building on CentOS 7, running on CentOS 6)

from docker-images.

Mostly it would work because the conda compilers build against CentOS 6 even on CentOS 7. Problems arise when you use hacks like https://github.com/conda-forge/cupy-feedstock/blob/master/recipe/meta.yaml#L28-L30 and it would no longer be usable in CentOS 6. (Of course if your dependencies don't work in CentOS 6, the package wouldn't either)

from docker-images.

Sorry, @isuruf, I still feel it's necessary to revisit your comment:

Why do you want to update the whole conda-forge system to CentOS 7 just to accommodate a proprietary software?

It is not just "one" proprietary software. As I mentioned earlier, it is the entire CUDA Toolkit (and everything built on top of it). The current docker images simply will not work with the upcoming CUDA 11.

Perhaps I am being naive, but I wonder if the following intermediate solution is possible while we continue the discussion: We set up two variants of docker images for CUDA 10.1 & 10.2 that is based on CentOS 7, and put them in production in conjunction with existing ones. Once CUDA 11 is out, we can simply add the support (so that from CUDA 11+ we use CentOS 7 only).

from docker-images.

Newer CUDA images use CentOS 7 (or in the most recent CentOS 8) as needed. Also older CUDA images had CentOS 7 variants added in PR ( #167 )

from docker-images.