intel / metrics-discovery Goto Github PK

Hi,
maybe good providing a simple sample of integrating this metrics API on a graphics/compute (OpenGL/OpenCL) app..
For ex. a OpenGL draw a single triangle app would be nice..

Supports all the APIs below?
*OpenGL Mesa..
*Vulkan Anvil driver
*OpenCL Beignet and new Neo OpenCL driver
*libVA video encode/decode usage

What counters on every API?
Thanks..

I recently enabled MDAPI support for the Intercept Layer for OpenCL Applications:

https://github.com/intel/opencl-intercept-layer

It's working, but I'm seeing a warning in the MDAPI build in our OSX CI build. The warning is:

/Users/travis/build/bashbaug/opencl-intercept-layer/mdapi/metrics_discovery_api.h:943:39: warning: 'MetricsDiscovery::IMetricSet_1_5::CalculateMetrics' hides overloaded virtual function [-Woverloaded-virtual]
        virtual TCompletionCode       CalculateMetrics( const unsigned char* rawData, uint32_t rawDataSize, TTypedValue_1_0* out,
                                      ^
/Users/travis/build/bashbaug/opencl-intercept-layer/mdapi/metrics_discovery_api.h:892:39: note: hidden overloaded virtual function 'MetricsDiscovery::IMetricSet_1_1::CalculateMetrics' declared here: different number of parameters (6 vs 7)
        virtual TCompletionCode       CalculateMetrics( const unsigned char* rawData, uint32_t rawDataSize, TTypedValue_1_0* out,
                                      ^
1 warning generated.

I've silenced this warning for now in the Intercept Layer for OpenCL Applications, but it'd be great if this were fixed in the official header file.

• OS

Operating System: Ubuntu 20.04 LTS
Kernel: Linux 5.4.0-26-generic
Architecture: x86-64

• error log:

/home/user/workspace/metrics-discovery/instrumentation/metrics_discovery/linux/md_driver_ifc_linux_perf.cpp:47:10: fatal error: xf86drm.h: No such file or directory
   47 | #include "xf86drm.h"       // for drmOpen/drmClose/drmIoctl
      |          ^~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/md.dir/build.make:271: CMakeFiles/md.dir/instrumentation/metrics_discovery/linux/md_driver_ifc_linux_perf.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:76: CMakeFiles/md.dir/all] Error 2
make: *** [Makefile:152: all] Error 2

Please, remove these 2 self-defined names required for the project build:

• MD_BUILD_TYPE (=release)
• MD_PLATFORM (=linux)

Instead use:

• CMAKE_BUILD_TYPE instead of MD_BUILD_TYPE
• CMAKE_SYSTEM_NAME instead of MD_PLATFORM

General rule: don't define self-names if the concept already exists. For example, defining MD_PLATFORM you step into issues with cross compiling which were already reported for media-drviver and mediasdk projects. If you need some other veriables to setup something, review https://cmake.org/Wiki/CMake_Cross_Compiling first.

Secondly, please, support default build configuration, i.e.:
cmake ..
You don't actually need to specify those 2 things you now require at all. You may wish to have a default for CMAKE_BUILD_TYPE as follows:

if(NOT CMAKE_BUILD_TYPE)
  message(STATUS "No build type selected, default to release")
  set(CMAKE_BUILD_TYPE release)
endif()

When using clang compiler -Wall is passed after all the skip -Wno exceptions.
I noticed passing -Wno-extern-c-compat using cmake doesn't suffice and instead the ordering if the compile options had to be altered.

This patch fixes the issue,

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 766d980..5386aad 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -392,6 +392,7 @@ elseif ("${PLATFORM}" STREQUAL "linux") # linux
         add_definitions(-D_DEBUG)
     endif ()

+    add_definitions(-Wall)
     add_definitions(-fdiagnostics-color=auto) # color=auto only to colorize compiler output
     add_definitions(-fstack-protector)
     add_definitions(-fvisibility=hidden)      # to disable exporting internal symbols (by default gcc exports all symbols)
@@ -416,7 +417,7 @@ elseif ("${PLATFORM}" STREQUAL "linux") # linux
     add_definitions(-DMD_USE_PERF)
     add_definitions(-DMETRICS_DISCOVERY)
     add_definitions(-fno-inline)
-    add_definitions(-Wall)
+    add_definitions(-Wno-extern-c-compat)

     # to enable x86 build on x64 host
     if ("${TARGET_ARCH}" STREQUAL "32")
@@ -592,4 +593,4 @@ elseif ("${PLATFORM}" STREQUAL linux)
     message ("INFO: CMAKE_CXX_FLAGS        = ${CMAKE_CXX_FLAGS}")
 else ()
     message ("-- Using platform is ${PLATFORM}")
-endif ()
\ No newline at end of file
+endif ()

Let me know if submitting a PR is better.

Hi all,

I use this tool, but I have a problem about executing sysmon(<pti-gpu/tools/sysmon>) not showing running process.
Just say "unknown". How can I do to fix this problem?

As the following, it is our experiment platform information:

Device 1-
CPU: i3-9100E(Coffee Lake)
GPU: Intel(R) UHD Graphics 630(GFL GT2)
OS: Ubuntu 22.04
Linux Kernel: 5.19.0-4-generic

Device 2-
CPU: i5-8500(coffee Lake)
GPU: Intel(R) Graphics(Coffeelake 3x8 GT2)
OS: Ubuntu 18.04
Linux Kernel: 4.15.0-136-generic

Can these two devices fully support the use of Intel's tool sysmon?

Thanks a lot.
Best regards,

uint64_t iu_strtoull( const char* str, char** endptr, int32_t base )
{
return strtoull( str, endptr, base );
}

is there ANY operating environment on this planet where strtoull is not strtoull ?

the abstraction layers in this project harm performance and obfuscate the code and will confuse static analysis and other compiler based security methods, making this project less secure

please seriously consider not doing pointless abstractions like this, a virtual kitten dies every time such abstraction is created

This issue is related to an issue filed by a user of the Intercept Layer for OpenCL Applications, which uses the metrics discovery features in Intel GPU drivers:

intel/opencl-intercept-layer#141

In short, when an application that is using time-based sampling crashes, abnormally exits, or (as described in the issue) ends in a Visual Studio debugging session, there is no opportunity to shut down and close the the metrics library properly. When the metrics library is not properly closed, time-based sampling remains enabled. This prevents subsequent applications from using time-based sampling. As best I can tell, there is no way to detect that this scenario has occurred or to recover from it.

Can the metrics library provide a mechanism to reset or recover in this scenario? It would be nice if the recovery were performed automatically, but if there were a mechanism to detect this state and explicitly request recovery I think that would work too.

Thanks!

Built library but did not find libmd.so in the install location since it's needed for Advisor/VTune to collect GPU metrics. Is there a way to obtain this lib file or is it the case that libigdmd needs to be renamed to libmd?

Hi all,

I use Intel tool gpuinfo which will use Metrics Discovery library.
And I follow the build and compile tutorial, I'm sure that I have already installed metrics discovery.
(Intel tool gpuinfo reference: https://github.com/intel/pti-gpu/tree/master/tools/gpuinfo)

But why I execute gpuinfo it doesn't show anything?
And it show hint that Metrics Discovery Application Programming Interface library is not found.
I don't know how to fix it.

OS:Ubuntu 22.04 (Linux kernel: 5.19.0-41)
CPU: i3-9100E (Coffee lake)
GPU: UHD Graphics 630

test@test:~/pti-gpu/tools/gpuinfo/build$ cmake -DCMAKE_BUILD_TYPE=Release ..

-- The CXX compiler identification is GNU 11.3.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Build Type: Relase
-- Bitness: 64 bits
-- Looking for C++ include metrics_discovery_api.h
-- Looking for C++ include metrics_discovery_api.h - found
CMake Warning at /home/test/pti-gpu/build_utils/CMakeLists.txt:339 (message):
  Intel(R) Metrics Discovery Application Programming Interface library is not
  found.  You may need to install it to fix this issue.
Call Stack (most recent call first):
  CMakeLists.txt:23 (CheckForMDLibrary)


-- Configuring done
-- Generating done
-- Build files have been written to: /home/test/pti-gpu/tools/gpuinfo/build

test@test:~/pti-gpu/tools/gpuinfo/build$ make

[ 50%] Building CXX object CMakeFiles/gpuinfo.dir/main.cc.o
[100%] Linking CXX executable gpuinfo
[100%] Built target gpuinfo

test@test:~/pti-gpu/tools/gpuinfo/build$ ./gpuinfo -l
Unable to find GPU devices
test@test:~/pti-gpu/tools/gpuinfo/build$ ./gpuinfo -i
Unable to find GPU devices

Thanks a lot.
Best regards,

All instructions for VTune point to enabling CONFIG_DRM_I915_LOW_LEVEL_TRACEPOINTS .
This doesn't seem like something that would work on a device that has iHD drivers installed from intel-media-driver

Hi i want documentation for intel metrics discover API.I want to know which performance counters quesries are being used for every metrics in Intel Graphics Performance Analyzer Metrics.Need help plz.

A customer got the following error while trying to compile the code:
In file included from <...>/metrics-discovery/instrumentation/utils/common/iu_debug.c:29:0:
<...>/metrics-discovery/instrumentation/utils/common/inc/iu_std.h:70:68: error: unknown type name 'va_list'

Is Mfx_usage supported similar to that in Intel PAL library?

Hi,

When I tried to build the source code, it takes very long(more than 5 minutes) to build the md_metrics_PVC_GT2.cpp file. Is it expected?

My machine is i9-12900K CPU.

Here's process information:

[openvino] mingyuki@dg2alderlake:~/local/openvino/bin/intel64/Release$ ps -ef | grep cc1plus
mingyuki 2742068 2742066 99 01:36 pts/14   00:03:39 /usr/lib/gcc/x86_64-linux-gnu/9/cc1plus -quiet -I /home/mingyuki/metrics-discovery/instrumentation/metrics_discovery/common/inc -I /home/mingyuki/metrics-discovery/instrumentation/metrics_discovery/common/inc/internal -I /home/mingyuki/metrics-discovery/external/drm -I /home/mingyuki/metrics-discovery/instrumentation/metrics_discovery/linux/inc -I /home/mingyuki/metrics-discovery/instrumentation/utils/common/inc -I /home/mingyuki/metrics-discovery/instrumentation/utils/linux/inc -I /home/mingyuki/metrics-discovery/inc/common -I /home/mingyuki/metrics-discovery/inc/common/instrumentation -I /home/mingyuki/metrics-discovery/inc/common/instrumentation/api -I /usr/include/libdrm -I /usr/include/libdrm/include/drm -I /usr/include/libdrm/intel -imultiarch x86_64-linux-gnu -D_GNU_SOURCE -D MD_USE_PERF -D METRICS_DISCOVERY -D NDEBUG -D _FORTIFY_SOURCE=2 -D metrics_discovery_EXPORTS /home/mingyuki/metrics-discovery/instrumentation/metrics_discovery/codegen/md_metrics_PVC_GT2.cpp -quiet -dumpbase md_metrics_PVC_GT2.cpp -msse -mtune=generic -march=x86-64 -auxbase-strip CMakeFiles/metrics_discovery.dir/instrumentation/metrics_discovery/codegen/md_metrics_PVC_GT2.cpp.o -O2 -Wno-reorder -Wformat=1 -Wno-unused -Wno-strict-aliasing -Wno-unknown-pragmas -Wno-comment -Wformat-security -Wall -std=c++11 -fdiagnostics-color=auto -fPIC -fstack-protector -fvisibility=hidden -fno-inline -fasynchronous-unwind-tables -Wformat-security -fstack-clash-protection -fcf-protection -o /tmp/ccpSfpBo.s



[openvino] mingyuki@dg2alderlake:~/local/openvino/bin/intel64/Release$ top
top - 01:41:17 up 26 days,  7:40,  1 user,  load average: 1.69, 2.74, 2.09
Tasks: 808 total,   2 running, 804 sleeping,   2 stopped,   0 zombie
%Cpu(s):  4.4 us,  0.2 sy,  0.0 ni, 95.4 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  64061.8 total,  32225.1 free,  13294.6 used,  18542.1 buff/cache
MiB Swap:   2048.0 total,    944.4 free,   1103.6 used.  49346.2 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
2742068 mingyuki  20   0 8251776   7.4g  21540 R 100.0  11.9   4:33.34 cc1plus

In the L3ShaderThroughput formulas for SKL, KBL, CFL, BXT, GLK there should be $$SlmBytesRead and $$SlmBytesWritten instead of $SlmBytesRead and $SlmBytesWritten.

"library has OS abstraction layer that allows for interoperability in different environments"..
but selecting any platform (MD_PLATFORM) other than Linux isn't currently supported (due to a check on md_cmake_adapter.cmake)..
also seeing Intel Windows driver has igdmd64 file and info mentions "Intel Metrics Discovery API" so seems this API is supported on Windows also..