mellanox / rshim-user-space Goto Github PK

Sorry for opening a ticket for this.

The Fedora Infrastructure has been trying to contact @lsun100 since September 5th without any success.
We raised the issue in the Fedora devel list (cf: https://lists.fedoraproject.org/archives/list/[email protected]/message/3NZNSFKSS25EVEA6RY42LVOPZ74NGUQU/ ), also without success.

@lsun100 if you see this, feel free to close it and reach out by email (either to me or to the address (admin@...) that is sending you the daily notifications).

Thanks!

I needed to make a modification to /etc/rshim.conf. After doing so, it seemed prudent to restart rshim.service. That took a very long time because rshim seems to ignore SIGTERM. Or perhaps it just ignores SIGTERM in the bad state that caused me to want to modify the configuration. journalctl tells me:

Aug 12 16:39:30 xxx systemd[1]: Stopping rshim driver for BlueField SoC...
Aug 12 16:41:00 xxx systemd[1]: rshim.service: State 'stop-sigterm' timed out. Killing.
Aug 12 16:41:00 xxx systemd[1]: rshim.service: Killing process 31084 (rshim) with signal SIGKILL.
Aug 12 16:41:00 xxx systemd[1]: rshim.service: Failed with result 'timeout'.
Aug 12 16:41:00 xxx systemd[1]: Stopped rshim driver for BlueField SoC.
Aug 12 16:41:00 xxx systemd[1]: Starting rshim driver for BlueField SoC...
Aug 12 16:41:00 xxx systemd[1]: Started rshim driver for BlueField SoC.

I observed similar problems with SIGINT when trying to use ^C to kill an rshim instance that was running in the foreground.

Note that this happened when I was debugging problems on a machine where the running kernel is missing cuse.ko (see #26), in case that matters. It was observed with the 2.0.5 build and with bits I built from commit 9b1da84.

Please consider using a simpler versioning scheme. The current version format ("2.0.6-19") that contains a dash is not friendly to downstream packaging. Distribution packages (e.g. Fedora, Debian, ...) use a dash as the separator between the upstream version and the distro packaging release fields.
Maybe semantic versioning (MAJOR.MINOR.PATCH)?: https://semver.org/

Dear team,

I'm working on Bluefield 2 DPU devices and have received scripts that can access low-level statistics. I don't know how much of the details are neccesary but in brief what the script does is open a file descriptor to rshim.

self.rshim_fd = os.open(filename, os.O_RDWR)

with filename /dev/rshim0
and then the script tries to

adapted_addr = (channel << 16) | addr
buf= array.array("B", 16*[0])                                                                                                               
struct.pack_into('<LQ',buf, 0, adapted_addr, 0)                                                                                             
sys.stdout.flush()                                                                                                                          
fcntl.ioctl(self.rshim_fd, 1, buf) 
val=buf[4:12]                                                                                                                               
rv=struct.unpack("<Q",bytearray(val))[0] 

which results in
IOError: [Errno 25] Inappropriate ioctl for device

I've looped through all the others possible values instead of 1 and the error is the same.
The version:

rshim -v
rshim 2.0.20

If you need any additional info about what the script does we can go deeper into it. I would be grateful for any help.

Best regards

When using dd to copy a bootstream to the rshim, I noticed that larger buffer sizes (e.g. 1MB) never finish copying. The last buffer always does not finish copying.

Example:
dd if=installer.bf of=/dev/rshim0/boot bs=1M conv=sync status=progress

When terminating dd with SIGINT, it shows the following:
435159040 bytes (435 MB, 415 MiB) copied, 80 s, 5.4 MB/s
^C
441+1 records in
441+0 records out
462422016 bytes (462 MB, 441 MiB) copied, 170.944 s, 2.7 MB/s

My bfb file was 442 MB large, so the last buffer is missing/never finished copying.

rshim has no chance of doing anything useful when the cuse kernel module cannot be loaded. It should log an appropriate message and exit non-zero immediately. This would make debugging this failure mode much easier.

This is with rshim built from commit 9b1da84.

# ./rshim -b pcie -d 0000:d8:00.2 -l 4 -f
modprobe: FATAL: Module cuse not found in directory /lib/modules/4.18.0-80.el8.27782638.x86_64

At this point it just keeps running.

Hi,

Would it be possible to add git tags for the new revisions?
git tag rshim-2.0.5-4 9fc3c34
git tag rshim-2.0.5-5 4d8571f

That would make integration in Open Build Service (https://build.opensuse.org/package/show/hardware/rshim) easier, as we could use a service file to download and check new software versions.