utcs-scea / altis Goto Github PK

A benchmarking suite for heterogeneous systems. The primary goal of this project is to improve and update aspects of existing benchmarking suites which are either insufficient or outdated.

Home Page: https://utcs-scea.github.io/altis/

License: BSD 2-Clause "Simplified" License

Makefile 0.58% Shell 0.84% Python 0.70% C++ 14.59% C 42.41% Cuda 40.14% CMake 0.74%

altis benchmarking-suite gpu cuda heterogeneity rodinia-benchmark-suite accelerator nvidia

altis's Introduction

Altis Benchmark Suite

Altis is a benchmark suite to test the performance and other aspects of systems with Graphics Processing Units (GPUs), developed in SCEA lab at University of Texas at Austin. Altis consists of a collection of GPU applications with differnt performance implications. Altis focuses primarily on Compute Unified Device Architecture (CUDA) computing platform.

Documentaion regarding this project can be found at the Wiki page. The Wiki document contains information regarding Altis setup, installation, usage, and other information.

We are refactoring Altis codebase for better usability and making it more developer-friendly. We made sure the benchmark still compile properly during refactoring so you can still use it. The refactoring involves changing how each benchmark application is used and adding more benchmarks.

How to Get Altis

Simply execute the following command:

git clone https://github.com/utcs-scea/altis.git

Setup

Altis relies on the avaialbility of CUDA and CMake (>= 3.8). Please refer to Environment Setup for how to set up Altis.

Build:

After the environment is setup properly, go to the root directory of Altis, execute:

./setup.sh

For more information regarding building process, please refer to Build for more information.

To Run a Single Benchmark:

cd src/cuda/$BENCHMARK_LEVEL/$BENCHMARK
./$BENCHMARK [options]

General Options: 
    -c, --configFile             specify configuration file
    -d, --device                 specify device to run on
    -i, --inputFile              path of input file
    -o, --outputFile             path of output file
    -m, --metricsFile            path of file to write metrics to
    -n, --passes                 specify number of passes
    -p, --properties             show properties for available platforms and devices (exits afterwards)
    -q, --quiet                  enable minimal output
    -s, --size                   specify problem size
    -v, --verbose                enable verbose output

Note: Run benchmark with --help to see full list of options available for that specific benchmark

To Run a Benchmark with Custom Data:

python data/$BENCHMARK/datagen.py [options]
Run benchmark with -i $DATA_FILEPATH

Note: Not all benchmarks have a datagen

Cite Us

Bibtex is shown below:

@INPROCEEDINGS{9238617,
author={B. {Hu} and C. J. {Rossbach}},
booktitle={2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)},
title={Altis: Modernizing GPGPU Benchmarks},
year={2020},
volume={},
number={},
pages={1-11},
doi={10.1109/ISPASS48437.2020.00011}}

Publication

B. Hu and C. J. Rossbach, "Altis: Modernizing GPGPU Benchmarks," 2020 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), Boston, MA, USA, 2020, pp. 1-11, doi: 10.1109/ISPASS48437.2020.00011.

Contact

For any questions regarding this project, please send an email to [email protected] or [email protected]

altis's People

Contributors

Stargazers

Watchers

Forkers

mkhairy yechen3 bodunhu jarusified hfingler shiyuw3 dazhuzhu-github yeonan pauldelestrac chr1s603 zoon17 tlimkim utcs-scea zzheng33

altis's Issues

With texture reference being removed from CUDA 12 version onwards, build gives errors like `DeviceMemory.cu(160): error: texture is not a template`

My system specifications:
GPU : NVIDIA GeForce RTX 2080
CUDA Version: 12.2

I tried the following steps:

$ git clone "https://github.com/utcs-scea/altis"
$ cd altis/
$ ./setup.sh
-- The CXX compiler identification is GNU 11.3.0
-- The CUDA compiler identification is NVIDIA 12.2.91
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done                                                                                       
-- Detecting CUDA compiler ABI info                                                                                            
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Configuring done (2.5s)
-- Generating done (0.1s)
-- Build files have been written to: /media/abhishek/Abhishek_HDD/workloads/altis/build
[  1%] Building CUDA object src/cuda/level0/devicememory/CMakeFiles/devicememoryLib.dir/DeviceMemory.cu.o
[  3%] Building CUDA object src/cuda/level0/busspeeddownload/CMakeFiles/busspeeddownloadLib.dir/BusSpeedDownload.cu.o
[  3%] Building CUDA object src/cuda/level0/maxflops/CMakeFiles/maxflopsLib.dir/MaxFlops.cu.o
[  3%] Building CXX object src/common/CMakeFiles/AltisCommon.dir/ResultDatabase.cpp.o
[  4%] Building CXX object src/common/CMakeFiles/AltisCommon.dir/OptionParser.cpp.o
[  5%] Building CUDA object src/cuda/level1/bfs/CMakeFiles/bfsLib.dir/bfs.cu.o
[  6%] Building CUDA object src/cuda/level0/busspeedreadback/CMakeFiles/busspeedreadbackLib.dir/BusSpeedReadback.cu.o
[  8%] Building CUDA object src/cuda/level2/cfd/CMakeFiles/cfddoubleLib.dir/euler3d_double.cu.o
[  9%] Building CUDA object src/cuda/level2/fdtd2d/CMakeFiles/fdtd2dLib.dir/fdtd2d.cu.o
[  8%] Building CUDA object src/cuda/level1/gemm/CMakeFiles/gemmLib.dir/Gemm.cu.o
[ 10%] Building CUDA object src/cuda/level1/gups/CMakeFiles/gupsLib.dir/gups.cu.o
[ 11%] Building CUDA object src/cuda/level1/pathfinder/CMakeFiles/pathfinderLib.dir/pathfinder.cu.o
[ 13%] Building CUDA object src/cuda/level2/cfd/CMakeFiles/cfdLib.dir/euler3d.cu.o
[ 14%] Building CUDA object src/cuda/level2/kmeans/CMakeFiles/kmeansLib.dir/kmmain.cu.o
[ 14%] Building CUDA object src/cuda/level1/sort/CMakeFiles/sortLib.dir/Sort.cu.o
[ 14%] Building CUDA object src/cuda/level2/dwt2d/CMakeFiles/dwt2dLib.dir/dwt_main.cu.o
/media/abhishek/Abhishek_HDD/workloads/altis/src/cuda/level0/devicememory/DeviceMemory.cu(160): error: texture is not a template
  texture<float4, 2, cudaReadModeElementType> texA;
  ^

/media/abhishek/Abhishek_HDD/workloads/altis/src/cuda/level0/devicememory/DeviceMemory.cu(510): error: identifier "cudaBindTextureToArray" is undefined
          __checkCudaErrors (cudaBindTextureToArray(texA, cuArray), "/media/abhishek/Abhishek_HDD/DGSF_workloads/altis/src/cuda/level0/devicememory/DeviceMemory.cu", 510);
                             ^

/media/abhishek/Abhishek_HDD/workloads/altis/src/cuda/level0/devicememory/DeviceMemory.cu(588): error: identifier "cudaUnbindTexture" is undefined
          __checkCudaErrors (cudaUnbindTexture(texA), "/media/abhishek/Abhishek_HDD/DGSF_workloads/altis/src/cuda/level0/devicememory/DeviceMemory.cu", 588);
                             ^

/media/abhishek/Abhishek_HDD/workloads/altis/src/cuda/level0/devicememory/DeviceMemory.cu(848): error: no instance of overloaded function "tex2D" matches the argument list
            argument types are: (<error-type>, float, float)
          float4 v = tex2D(texA, float(idx_x), float(idx_y));
                     ^

/media/abhishek/Abhishek_HDD/workloads/altis/src/cuda/level0/devicememory/DeviceMemory.cu(872): error: no instance of overloaded function "tex2D" matches the argument list
            argument types are: (<error-type>, float, float)
          float4 v = tex2D(texA, float(idx_x), float(idx_y));
                     ^

/media/abhishek/Abhishek_HDD/workloads/altis/src/cuda/level0/devicememory/DeviceMemory.cu(899): error: no instance of overloaded function "tex2D" matches the argument list
            argument types are: (<error-type>, float, float)
          float4 v = tex2D(texA, float(idx_x), float(idx_y));
                     ^

6 errors detected in the compilation of "/media/abhishek/Abhishek_HDD/workloads/altis/src/cuda/level0/devicememory/DeviceMemory.cu".
make[2]: *** [src/cuda/level0/devicememory/CMakeFiles/devicememoryLib.dir/build.make:77: src/cuda/level0/devicememory/CMakeFiles/devicememoryLib.dir/DeviceMemory.cu.o] Error 2
make[1]: *** [CMakeFiles/Makefile2:800: src/cuda/level0/devicememory/CMakeFiles/devicememoryLib.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 15%] Building CUDA object src/cuda/level2/dwt2d/CMakeFiles/dwt2dLib.dir/dwt.cu.o
[ 16%] Building CXX object src/common/CMakeFiles/AltisCommon.dir/Option.cpp.o
[ 17%] Building CXX object src/common/CMakeFiles/AltisCommon.dir/ProgressBar.cpp.o
[ 18%] Building CUDA object src/cuda/level2/dwt2d/CMakeFiles/dwt2dLib.dir/components.cu.o
[ 19%] Linking CXX static library libAltisCommon.a
[ 19%] Built target AltisCommon
[ 20%] Building CUDA object src/cuda/level1/sort/CMakeFiles/sortLib.dir/sort_kernel.cu.o
[ 21%] Linking CUDA static library libbusspeedreadbackLib.a
[ 21%] Built target busspeedreadbackLib
[ 22%] Building CUDA object src/cuda/level2/dwt2d/CMakeFiles/dwt2dLib.dir/dwt_cuda/fdwt53.cu.o
[ 22%] Linking CUDA static library libbusspeeddownloadLib.a
[ 22%] Built target busspeeddownloadLib
[ 23%] Building CUDA object src/cuda/level2/dwt2d/CMakeFiles/dwt2dLib.dir/dwt_cuda/rdwt53.cu.o
[ 24%] Building CUDA object src/cuda/level2/dwt2d/CMakeFiles/dwt2dLib.dir/dwt_cuda/fdwt97.cu.o
[ 25%] Linking CUDA static library libsortLib.a
[ 26%] Building CUDA object src/cuda/level2/dwt2d/CMakeFiles/dwt2dLib.dir/dwt_cuda/rdwt97.cu.o
[ 26%] Built target sortLib
[ 26%] Linking CUDA static library libpathfinderLib.a
[ 26%] Building CUDA object src/cuda/level2/dwt2d/CMakeFiles/dwt2dLib.dir/dwt_cuda/common.cu.o
[ 27%] Linking CUDA static library libgupsLib.a
[ 28%] Linking CUDA static library libbfsLib.a
[ 28%] Built target pathfinderLib
[ 28%] Built target gupsLib
[ 28%] Building CUDA object src/cuda/level2/kmeans/CMakeFiles/kmeansLib.dir/kmeansraw.cu.o
[ 28%] Built target bfsLib
[ 29%] Linking CUDA static library libcfddoubleLib.a
[ 30%] Linking CUDA static library libcfdLib.a
[ 30%] Built target cfddoubleLib
[ 30%] Built target cfdLib
[ 31%] Linking CUDA static library libgemmLib.a
[ 31%] Built target gemmLib
[ 31%] Linking CUDA static library libfdtd2dLib.a
[ 31%] Built target fdtd2dLib
[ 32%] Linking CUDA static library libdwt2dLib.a
[ 32%] Built target dwt2dLib
[ 33%] Linking CUDA static library libmaxflopsLib.a
[ 33%] Built target maxflopsLib
[ 34%] Linking CUDA static library libkmeansLib.a
[ 34%] Built target kmeansLib
make: *** [Makefile:91: all] Error 2

The errors are because NVIDIA has dropped the support for texture references from CUDA 12.0, mentioned here.

Build Failed on Ubuntu20.04

Hi, I'm trying to build Altis on my server and on docker container, but both encounter the same errors. The following descriptions only show a part of the error messages. here contains the complete error messages

Environment

Ubuntu: 20.04
CUDA version: 11.8
Docker image: nvidia/cuda:11.8.0-devel-ubuntu20.04
cmake: 3.16
GPU: nvidia A100, sm number: 80

Error Messages

First I tried to run ./setup.sh and saw the following result

Then I tried to understand the building process, so I checked here and applied these steps manually.
When running cmake -DCMAKE_CUDA_ARCHITECTURES=80 it shows the following message. But I'm not sure if this is important.

The fatal error occurred when running the last make command.

Reproduce Steps

Run nvidia docker image

sudo nvidia-docker run -it nvidia/cuda:11.8.0-devel-ubuntu20.04 /bin/bash

Install git and cmake

apt-get update
apt-get install git cmake

Clone this repo

git clone https://github.com/utcs-scea/altis.git

run setup.sh or follow the build steps to build Altis.

Thanks for viewing my issue. Any reply is appreciated

ptxas fatal : Value 'sm_30' is not defined for option 'gpu-name'

I'm using gtx1650 on my computer, with cuda 11.6, cmake 3.13.0. How can i solve the problem

Run Build Command:"/usr/bin/make" "cmTC_a1746/fast"
/usr/bin/make -f CMakeFiles/cmTC_a1746.dir/build.make CMakeFiles/cmTC_a1746.dir/build
make[1]: Entering directory '/home/ming/gpgpu/altis/CMakeFiles/CMakeTmp'
Building CUDA object CMakeFiles/cmTC_a1746.dir/main.cu.o
/usr/bin/nvcc -x cu -c /home/ming/gpgpu/altis/CMakeFiles/CMakeTmp/main.cu -o CMakeFiles/cmTC_a1746.dir/main.cu.o
ptxas fatal : Value 'sm_30' is not defined for option 'gpu-name'
make[1]: *** [CMakeFiles/cmTC_a1746.dir/build.make:66: CMakeFiles/cmTC_a1746.dir/main.cu.o] Error 255
make[1]: Leaving directory '/home/ming/gpgpu/altis/CMakeFiles/CMakeTmp'
make: *** [Makefile:121: cmTC_a1746/fast] Error 2

question about a level2 benchmark

I tried to build the benchmark and the compiler reports the errors

cd yolo/src
nvcc -I../include -O3 *.c *.cu

compare.c: In function ‘train_compare’:
compare.c:17:19: error: invalid initializer
   17 |     network net = parse_network_cfg(cfgfile);
      |                   ^~~~~~~~~~~~~~~~~
compare.c:54:36: error: incompatible type for argument 1 of ‘train_network’
   54 |         float loss = train_network(net, train);
      |                                    ^~~
      |                                    |
      |                                    network {aka struct network}
In file included from network.h:4,
                 from compare.c:3:
../include/darknet.h:764:30: note: expected ‘network *’ {aka ‘struct network *’} but argument is of type ‘network’ {aka ‘struct network’}
  764 | float train_network(network *net, data d);
      |                     ~~~~~~~~~^~~
compare.c:62:26: error: incompatible type for argument 1 of ‘save_weights’
   62 |             save_weights(net, buff);
      |                          ^~~
      |                          |
      |                          network {aka struct network}
In file included from network.h:4,
                 from compare.c:3:
../include/darknet.h:683:28: note: expected ‘network *’ {aka ‘struct network *’} but argument is of type ‘network’ {aka ‘struct network’}
  683 | void save_weights(network *net, char *filename);
      |                   ~~~~~~~~~^~~
compare.c:69:26: error: incompatible type for argument 1 of ‘save_weights’
   69 |             save_weights(net, buff);
      |                          ^~~
      |                          |
      |                          network {aka struct network}
In file included from network.h:4,
                 from compare.c:3:
../include/darknet.h:683:28: note: expected ‘network *’ {aka ‘struct network *’} but argument is of type ‘network’ {aka ‘struct network’}
  683 | void save_weights(network *net, char *filename);
      |                   ~~~~~~~~~^~~
compare.c:75:18: error: incompatible type for argument 1 of ‘free_network’
   75 |     free_network(net);
      |                  ^~~
      |                  |
      |                  network {aka struct network}
In file included from network.h:4,
                 from compare.c:3:
../include/darknet.h:691:28: note: expected ‘network *’ {aka ‘struct network *’} but argument is of type ‘network’ {aka ‘struct network’}
  691 | void free_network(network *net);
      |                   ~~~~~~~~~^~~
compare.c: In function ‘validate_compare’:
compare.c:84:19: error: invalid initializer
   84 |     network net = parse_network_cfg(filename);
      |                   ^~~~~~~~~~~~~~~~~
compare.c:130:44: error: incompatible type for argument 1 of ‘network_predict_data’
  130 |         matrix pred = network_predict_data(net, val);
      |                                            ^~~
      |                                            |
      |                                            network {aka struct network}
In file included from network.h:4,
                 from compare.c:3:
../include/darknet.h:737:38: note: expected ‘network *’ {aka ‘struct network *’} but argument is of type ‘network’ {aka ‘struct network’}
  737 | matrix network_predict_data(network *net, data test);
      |                             ~~~~~~~~~^~~
compare.c: In function ‘bbox_comparator’:
compare.c:182:42: error: incompatible type for argument 1 of ‘network_predict’
  182 |     float *predictions = network_predict(net, X);
      |                                          ^~~
      |                                          |
      |                                          network {aka struct network}
In file included from network.h:4,
                 from compare.c:3:
../include/darknet.h:740:33: note: expected ‘network *’ {aka ‘struct network *’} but argument is of type ‘network’ {aka ‘struct network’}
  740 | float *network_predict(network *net, float *input);
      |                        ~~~~~~~~~^~~
compare.c: In function ‘bbox_fight’:
compare.c:211:42: error: incompatible type for argument 1 of ‘network_predict’
  211 |     float *predictions = network_predict(net, X);
      |                                          ^~~
      |                                          |
      |                                          network {aka struct network}
In file included from network.h:4,
                 from compare.c:3:
../include/darknet.h:740:33: note: expected ‘network *’ {aka ‘struct network *’} but argument is of type ‘network’ {aka ‘struct network’}
  740 | float *network_predict(network *net, float *input);
      |                        ~~~~~~~~~^~~
compare.c: In function ‘SortMaster3000’:
compare.c:230:19: error: invalid initializer
  230 |     network net = parse_network_cfg(filename);
      |                   ^~~~~~~~~~~~~~~~~
compare.c: In function ‘BattleRoyaleWithCheese’:
compare.c:262:19: error: invalid initializer
  262 |     network net = parse_network_cfg(filename);
      |                   ^~~~~~~~~~~~~~~~~

error: ‘stoi’ is not a member of ‘std’

My enviroment is win11, wsl2, g++,gcc 4.4.7
I met the problem :

deviceQuery.cpp: In function ‘int main(int, char**)’:
deviceQuery.cpp:176: error: ‘stoi’ is not a member of ‘std’
deviceQuery.cpp:180: error: no matching function for call to ‘std::basic_fstream<char, std::char_traits >::open(std::string&, std::_Ios_Openmode)’
/usr/include/c++/4.4/fstream:865: note: candidates are: void std::basic_fstream<_CharT, _Traits>::open(const char*, std::_Ios_Openmode) [with _CharT = char, _Traits = std::char_traits]
make: *** [Makefile:326: deviceQuery.o] Error 1
Failed to compile deviceQuery

I try the solution on stackoverflow but don't work
https://stackoverflow.com/questions/38034197/compile-error-stoi-is-not-a-member-of-std

Softmax benchmark

Hi
I don't see DNN workloads in level 0, 1 and 2 folders. I mean softmax, activation and ...
Where should I look exactly?

GEMM out of memory

./gemm -s 5 --passes 30 --uvm

Chose device: name='NVIDIA GeForce RTX 3090' index=0
Running GEMM
Running single precision test
Running double precision test
Running half preicsion test
/workspace/benchmark/altis/src/cuda/level1/gemm/Gemm.cu(238) : CUDA Runtime API error 2: out of memory.