Benchmarks on the Turing Pi Cluster about turing-pi-cluster HOT 10 CLOSED

Brad Manske posted this comment on my YouTube channel:

I didn't even notice—the two Inateck external cases I had were both the older 'non-UASP' type (link on Amazon — note the two options, with/without UASP). So I ordered a UASP type case and will be updating the benchmarks above (and in the next video).

Edit: Holy cow, big improvement!

To confirm that the drive is using UASP, check with lsusb -t:

$ lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 1: Dev 2, If 0, Class=Mass Storage, Driver=uas, 5000M

It shows Driver=uas for UASP-enabled drives (USB Attached SCSI Protocol), and usb-storage for non-UASP (Bulk-Only Transport/BOT).

Also, regarding power consumption:

• without UASP: 5.09v at 1.04A - 5.29W (Idle 5.07v @ 0.74A)
• with UASP: 5.13v at 0.95A - 4.87W (Idle 5.07v @ 0.67A)
• % difference: 8% power savings

Note: Power measurements taken with Pi 4 headless with Pi-FAN plugged in via GPIO (uses ~0.20A), Ethernet, and the USB 3.0 drive. Used a Satechi USB-C power tester.

Good article on UASP from 2015: What Is A UASP Storage Enclosure?.

from turing-pi-cluster.

Caveats: The Turing Pi cluster is running 7 Compute Modules, CM3+ boards with 1 GB of RAM each. The Pi Dramble cluster is running 4 Pi 4 2 GB models. The two clusters used the same exact K3s setup and Turing Pi Cluster configurations.

from turing-pi-cluster.

I did a 15 minute burn-in (running wrk on both the Drupal and Wordpress sites simultaneously) and it seems like the Pis were able to keep their cool—just barely.

Guess which one is worker-03?

from turing-pi-cluster.

Running the same burn-in test on the Dramble cluster (with the same configuration) now. I'll try to remember to grab an IR image as well, 10 minutes in or so.

One interesting takeaway: Wordpress is surprisingly CPU intense in its default config for non-authed users, whereas Drupal is way lighter and can serve up 2x the requests using its default caching mechanisms. They're equal when you throw a caching proxy in front or dump to HTML.

from turing-pi-cluster.

So... I just re-tested the Dramble cluster running Raspberry Pi OS's 64-bit beta version, and while Wordpress and Minecraft were faster, Drupal was slower. I'm trying to see if maybe something in Drupal core got way slower in the most recent point release (doubtful), or if maybe Hypriot vs Raspberry Pi OS 32-bit might have some performance issue that Drupal runs into.

To be clear: I'm running the exact same hardware (4 2 GB Pi 4s), with the exact same microSD cards (physically—I am re-flashing the same cards with different OSes so it can't be a card-to-card variance), and running all tests at least 3 times allowing a 30 minute warm-up period before starting.

(Benchmarking is hard, but turns up interesting results sometimes.)

from turing-pi-cluster.

I'm also seeing a fair bit of variance in how long it takes Minecraft to generate a new world... so I'm going to have to re-test that a bit.

I'm guessing the reason Drupal/Wordpress are slower is because on a memory-constrained 64-bit OS, you can't squash as many workers into the same amount of memory, so you might be able to have 3 or 4 workers on a 512 MB instance running Drupal on 32-bit, whereas you can only fit 2 or 3 on 64-bit due to pointers being larger, and these PHP applications storing a lot of tiny bits of state in memory.

So 64-bit benefits purely-CPU-driven processing (e.g. encodes and such—see geerlingguy/drupal-pi#45), and processes where you have massive amounts of RAM available (e.g. 4 GB or more), but actually presents a bit of a limit with highly-memory-constrained environments (e.g. < 2 GB of RAM).

from turing-pi-cluster.

Just noting that since I'm sure someone may ask—yes, you can max out the bandwidth of all the Turing Pi nodes at the same time, and the onboard Gigabit switch will handle the traffic. As an example, I set up a connection to two pis and ran iperf to both of them from my Mac (which gets ~930 Mbps to a single Pi 4) at the same time:

$ iperf -c 10.0.100.163 & iperf -c 10.0.100.74
[1] 19606
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 10.0.100.74, TCP port 5001
Client connecting to 10.0.100.163, TCP port 5001
TCP window size:  129 KByte (default)
------------------------------------------------------------
TCP window size:  129 KByte (default)
------------------------------------------------------------
[  4] local 10.0.100.118 port 54978 connected with 10.0.100.74 port 5001
[  4] local 10.0.100.118 port 54977 connected with 10.0.100.163 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   113 MBytes  94.7 Mbits/sec
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   112 MBytes  93.9 Mbits/sec
[1]  + done       iperf -c 10.0.100.163

I could confirm that the Mac's network stats showed double the throughput during the test (~24 MB/sec when doing both, vs ~12 MB/sec when doing just one).

from turing-pi-cluster.

Leaving open just so I can re-run the Minecraft world generation benchmark again in the 3 scenarios where I didn't record the correct results.

from turing-pi-cluster.

Does that same improvement seen on the USB 3.0-native Pi 4 translate at all to USB 2.0 ports on the CM3+?

It was mounted with usb-storage driver when I tested with the CM PoE Board:

Port 4: Dev 4, If 0, Class=Mass Storage, Driver=usb-storage, 480M

Some unofficial confirmation that the BCM2835 doesn't support UASP because it lacks 'scatter gather' which is a requirement of the Linux UASP driver.

from turing-pi-cluster.

Benchmarking episode is live (Episode 5 - Benchmarking the Turing Pi), and I referenced it in the README in this commit: 6191dd7

I will likely be doing a little more disk IO testing later as I have some new hardware coming in, but that can wait for a new issue... this one's already a bit overloaded!

from turing-pi-cluster.