Giter VIP home page Giter VIP logo

dpu_demo's Introduction

UPMEM Logo

The UPMEM Processing-In-Memory solution consists of a pool of programmable co-processors, the DPUs, integrated in the memory.

Content

Examples of DPU programs generated with UPMEM DPU SDK

Requirements

Building the programs

make to build the programs.

make clean removes the resulting builds.

Executing

The simplest way to execute the programs is to run make test. Otherwise, you may run individual tests manually by executing the binary generated in the build folder of the targeted program.

List of demo programs

checksum

The host application generates a random file of 8MB, and schedules multiple DPUs co-processors to compute the checksum on this file. As sanity check it compares host result with the DPU results, and prints out the DPU performances:

  • Total number of DPU cycles executed
  • The number of DPU cycles per byte processed

Macros can be used to change the dimensions of the program:

  • NR_TASKLETS set the number of tasklets used by the dpu program (default: 16)
  • NR_DPUS set the number of dpus used by the host program (default: 1)

example: make test NR_TASKLETS=1 NR_DPUS=DPU_ALLOCATE_ALL

Two implementations of the Host application are provided:

  • host.c using the C Host API
  • host.py using the Python Host API

make test will run both implementations. Use the Makefile targets test_c or test_python to specify an implementation.

Note: because the input file is randomly generated, the checksum may differ between the two implementations.

SDK Documentation

https://sdk.upmem.com/stable/

dpu_demo's People

Contributors

dlagneux-upmem avatar furodet avatar gfroid avatar jordichauzi avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

dpu_demo's Issues

rebuild properly when build parameters have changed

When using the build defines on the command-line as documented in the example:
NR_TASKLETS=1 NR_DPUS=DPU_ALLOCATE_ALL make test
we need to ensure that the binary is rebuilt when needed (ie parameters have changed)

Issue using the dpu profiling tool

Dear UPMEM Tech support,

I have been grated acess to the server but Iโ€™m still working on the emulator. I was trying to run the DPU profiling tool and got the following errors:

When trying to run the following command:
sudo dpu-profiling functions -o chrometf.json -A -a -o checksum_host_optimized.vcf -- ./checksum_host_optimized

This following error appears:
Failed executing: ['perf', 'data', 'convert', '--to-chrometf', 'out_tmp.json', '--all', '--input', 'perf.data']

I tried to run the command that the python script is trying to execute and got the following:
No conversion support compiled in. perf should be compiled with environment variables LIBBABELTRACE=1 and LIBBABELTRACE_DIR=/path/to/libbabeltrace/

I tried multiple fixes, however the one that made the most sense was to build perf linked with babeltrace compiling perf with those enviroment variables. I installed babeltrace using sudo apt-get install babeltrace

So I navigated to the perf folder:

pribeiro@RTX3095:/home$ cd /linux/tools/perf

And tried to compile perf using the following command:
sudo LIBBABELTRACE=1 LIBBABELTRACE_DIR=/usr/bin/babeltrace/ make

This was the output of the make (relevant parts):

pribeiro@RTX3095:~/linux/tools/perf$ sudo LIBBABELTRACE=1 LIBBABELTRACE_DIR=/usr/bin/ make
BUILD: Doing 'make -j48' parallel build
Warning: Kernel ABI header at 'tools/include/uapi/drm/i915_drm.h' differs from latest version at 'include/uapi/drm/i915_drm.h'
diff -u tools/include/uapi/drm/i915_drm.h include/uapi/drm/i915_drm.h
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h'
diff -u tools/include/uapi/linux/kvm.h include/uapi/linux/kvm.h
Warning: Kernel ABI header at 'tools/include/uapi/linux/vhost.h' differs from latest version at 'include/uapi/linux/vhost.h'
diff -u tools/include/uapi/linux/vhost.h include/uapi/linux/vhost.h
Warning: Kernel ABI header at 'tools/arch/x86/include/asm/disabled-features.h' differs from latest version at 'arch/x86/include/asm/disabled-features.h'
diff -u tools/arch/x86/include/asm/disabled-features.h arch/x86/include/asm/disabled-features.h
Warning: Kernel ABI header at 'tools/arch/x86/include/asm/cpufeatures.h' differs from latest version at 'arch/x86/include/asm/cpufeatures.h'
diff -u tools/arch/x86/include/asm/cpufeatures.h arch/x86/include/asm/cpufeatures.h
Warning: Kernel ABI header at 'tools/arch/arm64/include/uapi/asm/perf_regs.h' differs from latest version at 'arch/arm64/include/uapi/asm/perf_regs.h'
diff -u tools/arch/arm64/include/uapi/asm/perf_regs.h arch/arm64/include/uapi/asm/perf_regs.h
Warning: Kernel ABI header at 'tools/arch/x86/include/uapi/asm/svm.h' differs from latest version at 'arch/x86/include/uapi/asm/svm.h'
diff -u tools/arch/x86/include/uapi/asm/svm.h arch/x86/include/uapi/asm/svm.h
Warning: Kernel ABI header at 'tools/arch/arm64/include/uapi/asm/kvm.h' differs from latest version at 'arch/arm64/include/uapi/asm/kvm.h'
diff -u tools/arch/arm64/include/uapi/asm/kvm.h arch/arm64/include/uapi/asm/kvm.h
Warning: Kernel ABI header at 'tools/include/linux/coresight-pmu.h' differs from latest version at 'include/linux/coresight-pmu.h'
diff -u tools/include/linux/coresight-pmu.h include/linux/coresight-pmu.h
Makefile.config:459: No libdw DWARF unwind found, Please install elfutils-devel/libdw-dev >= 0.158 and/or set LIBDW_DIR
Makefile.config:464: No libdw.h found or old libdw.h found or elfutils is older than 0.138, disables dwarf support. Please install new elfutils-devel/libdw-dev
Makefile.config:617: DWARF support is off, BPF prologue is disabled
Makefile.config:920: Old version of libbfd/binutils things like PE executable profiling will not be available
Makefile.config:982: No libzstd found, disables trace compression, please install libzstd-dev[el] and/or set LIBZSTD_DIR
Makefile.config:993: No libcap found, disables capability support, please install libcap-devel/libcap-dev
Makefile.config:1006: No numa.h found, disables 'perf bench numa mem' benchmark, please install numactl-devel/libnuma-devel/libnuma-dev
Makefile.config:1090: No alternatives command found, you need to set JDIR= to point to the root of your Java directory

Auto-detecting system features:
... dwarf: [ OFF ]
... dwarf_getlocations: [ OFF ]
... glibc: [ on ]
... libbfd: [ OFF ]
... libbfd-buildid: [ OFF ]
... libcap: [ OFF ]
... libelf: [ on ]
... libnuma: [ OFF ]
... numa_num_possible_cpus: [ OFF ]
... libperl: [ on ]
... libpython: [ on ]
... libcrypto: [ on ]
... libunwind: [ on ]
... libdw-dwarf-unwind: [ OFF ]
... zlib: [ on ]
... lzma: [ on ]
... get_cpuid: [ on ]
... bpf: [ on ]
... libaio: [ on ]
... libzstd: [ OFF ]
... disassembler-four-args: [ OFF ]

CC pmu-events/pmu-events.o
make[3]: Nothing to be done for 'install_headers'.

After, I ran make install:

pribeiro@RTX3095:~/linux/tools/perf$ sudo LIBBABELTRACE=1 LIBBABELTRACE_DIR=/usr/bin/ make install
BUILD: Doing 'make -j48' parallel build
Warning: Kernel ABI header at 'tools/include/uapi/drm/i915_drm.h' differs from latest version at 'include/uapi/drm/i915_drm.h'
diff -u tools/include/uapi/drm/i915_drm.h include/uapi/drm/i915_drm.h
Warning: Kernel ABI header at 'tools/include/uapi/linux/kvm.h' differs from latest version at 'include/uapi/linux/kvm.h'
diff -u tools/include/uapi/linux/kvm.h include/uapi/linux/kvm.h
Warning: Kernel ABI header at 'tools/include/uapi/linux/vhost.h' differs from latest version at 'include/uapi/linux/vhost.h'
diff -u tools/include/uapi/linux/vhost.h include/uapi/linux/vhost.h
Warning: Kernel ABI header at 'tools/arch/x86/include/asm/disabled-features.h' differs from latest version at 'arch/x86/include/asm/disabled-features.h'
diff -u tools/arch/x86/include/asm/disabled-features.h arch/x86/include/asm/disabled-features.h
Warning: Kernel ABI header at 'tools/arch/x86/include/asm/cpufeatures.h' differs from latest version at 'arch/x86/include/asm/cpufeatures.h'
diff -u tools/arch/x86/include/asm/cpufeatures.h arch/x86/include/asm/cpufeatures.h
Warning: Kernel ABI header at 'tools/arch/arm64/include/uapi/asm/perf_regs.h' differs from latest version at 'arch/arm64/include/uapi/asm/perf_regs.h'
diff -u tools/arch/arm64/include/uapi/asm/perf_regs.h arch/arm64/include/uapi/asm/perf_regs.h
Warning: Kernel ABI header at 'tools/arch/x86/include/uapi/asm/svm.h' differs from latest version at 'arch/x86/include/uapi/asm/svm.h'
diff -u tools/arch/x86/include/uapi/asm/svm.h arch/x86/include/uapi/asm/svm.h
Warning: Kernel ABI header at 'tools/arch/arm64/include/uapi/asm/kvm.h' differs from latest version at 'arch/arm64/include/uapi/asm/kvm.h'
diff -u tools/arch/arm64/include/uapi/asm/kvm.h arch/arm64/include/uapi/asm/kvm.h
Warning: Kernel ABI header at 'tools/include/linux/coresight-pmu.h' differs from latest version at 'include/linux/coresight-pmu.h'
diff -u tools/include/linux/coresight-pmu.h include/linux/coresight-pmu.h
Makefile.config:459: No libdw DWARF unwind found, Please install elfutils-devel/libdw-dev >= 0.158 and/or set LIBDW_DIR
Makefile.config:464: No libdw.h found or old libdw.h found or elfutils is older than 0.138, disables dwarf support. Please install new elfutils-devel/libdw-dev
Makefile.config:617: DWARF support is off, BPF prologue is disabled
Makefile.config:920: Old version of libbfd/binutils things like PE executable profiling will not be available
Makefile.config:982: No libzstd found, disables trace compression, please install libzstd-dev[el] and/or set LIBZSTD_DIR
Makefile.config:993: No libcap found, disables capability support, please install libcap-devel/libcap-dev
Makefile.config:1006: No numa.h found, disables 'perf bench numa mem' benchmark, please install numactl-devel/libnuma-devel/libnuma-dev
Makefile.config:1090: No alternatives command found, you need to set JDIR= to point to the root of your Java directory

Auto-detecting system features:
... dwarf: [ OFF ]
... dwarf_getlocations: [ OFF ]
... glibc: [ on ]
... libbfd: [ OFF ]
... libbfd-buildid: [ OFF ]
... libcap: [ OFF ]
... libelf: [ on ]
... libnuma: [ OFF ]
... numa_num_possible_cpus: [ OFF ]
... libperl: [ on ]
... libpython: [ on ]
... libcrypto: [ on ]
... libunwind: [ on ]
... libdw-dwarf-unwind: [ OFF ]
... zlib: [ on ]
... lzma: [ on ]
... get_cpuid: [ on ]
... bpf: [ on ]
... libaio: [ on ]
... libzstd: [ OFF ]
... disassembler-four-args: [ OFF ]

Makefile:210: Please install asciidoc xmlto to have the man pages installed
make[3]: Nothing to be done for 'try-install-man'.
make[3]: Nothing to be done for 'install_headers'.
DESCEND plugins
INSTALL trace_plugins
INSTALL binaries
INSTALL tests
INSTALL libexec
INSTALL bpf-headers
INSTALL bpf-examples
INSTALL perf-archive
INSTALL perf-iostat
INSTALL strace/groups
INSTALL perl-scripts
INSTALL python-scripts
INSTALL dlfilters
INSTALL perf_completion-script
INSTALL perf-tip

I tried to re-run the profiler and then the perf command and got the same erros as before.

Would you be so kind as to help me solve this issue?

dpu allocation error

Hi all,

I'm struggling to run this demo on real UPMEM. After issuing "make test", I keep getting DPU Error (dpu allocation error). The error occurs in this line. Does anyone have an idea how to troubleshoot this?

Cheers,
Hamid

how to use dpu-lldb

plz help me! I followed steps in this website, but it didn't work. https://sdk.upmem.com/2023.2.0/02_HelloWorld.html
I have installed the upmem sdk tar.gz file and set my variables. But I can't execute the helloworld.c after dpu-upmem-dpurte-clang -o helloworld helloworld.c. If I execute file helloworld, it shows the architecture is not captible. How can I resolve this problem?

Question on dpu-lldb

Hi,
First of all, I am sorry if this is not a good place to ask this.

I encountered an error using dpu-lldb.
I follow the sdk documentation to write the helloworld.c program for DPU, compile it, and run it in dpu-lldb.
The package I use is the deb package for SDK version 2020.3.0 on my own Ubuntu 18.04 server (no DPUs installed).

However, when I execute process launch, I got the following error:

error: process launch failed: 'A' packet returned an error: 8

Hope you can help me identify the problems.
Thank you!

Sincerely,
Kuo-Feng

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.