cublaslt_gemm fp8 does not work with RTX 40 about superbenchmark HOT 4 CLOSED

Sorry. SuperBenchmark doesn't officially support GeForce series at all. It would be great if you would like to contribute to it. Besides, we also don't know whether RTX 4090 supports FP8 or not as well.

from superbenchmark.

Hi @edisonchan, according to NVIDIA's cuBLASLt document:

FP8 was first introduced with Hopper GPUs (compute capability 9.0 or above) and is designed to further accelerate matrix multiplications.

RTX 4090 is built on Ada Lovelace arch with compute capability 8.9, which does not meet above requirement to use cuBLASLt FP8 currently. You can also check this discussion.

from superbenchmark.

Hi @edisonchan, according to NVIDIA's cuBLASLt document:

FP8 was first introduced with Hopper GPUs (compute capability 9.0 or above) and is designed to further accelerate matrix multiplications.

RTX 4090 is built on Ada Lovelace arch with compute capability 8.9, which does not meet above requirement to use cuBLASLt FP8 currently. You can also check this discussion.

I am aware of that discussion, but according to another discussion on transformer-engine, the latest version of cudatoolkit should provide support. NVIDIA/TransformerEngine#15

from superbenchmark.

the latest version of cudatoolkit should provide support.

CUDA 12.1 update 1 was just released several days ago, and the nvcr container 23.04 is not released yet. You may run it if manually build the benchmark source with latest CUDA 12.1.1, we don't have RTX devices to try.

These versions won't be upgraded frequently neither since the main focus is on data-center GPUs.

from superbenchmark.