Feature request/question on EDR support for FS. about azurehpc HOT 7 OPEN

I’m not independently measuring disk performance, but trying to run a performance benchmark on a cluster. My reason for thinking I need better performance is simply that a single node run done on the BeeGFS filesystem I initially set up performed much worse than one run from a localdisk. (And the multinode runs were correspondingly bad). So, I will try a Lustre filesystem and want to try to make it as good as possible without spending a week on doing so. I did not know the nodes needed to be in the same scaleset to communicate over IB. That is very helpful. Thank you! I’m still not sure I want to do that, as, obviously, the I/O traffic would interfere with internode communication. I don’t know the application’s behavior well enough to judge the relative traffic levels of I/O and message passing. (In an ideal world, yes, I would, but so it goes.) Would just increasing the number of LFS OSS servers be a second best way to improve the PFS I/O performance? My capacity need for this filesystem are actually pretty small, it’s all about the performance, so maybe, if I do try to use the IB interface, using /dev/sdb on some of the IB enabled node types should be sufficient. In the config.json, I would guess that I would only need, after changing the vmtype, to change the nvme entry to sdb here: { "script": "lfsoss.sh", "args": [ "$(<hostlists/tags/lfsmaster)", "/dev/nvme0n1" ], "tag": "lfsoss", "sudo": true }, But I don’t know how to make it use the IB interface. Would I have to go in and do that after the fact, or can AzureHPC scripting do that, too? Of course, I would still have to figure out how to get both OSS and compute in the same scale set. Thank you for any suggestions. Tim From: edwardsp <[email protected]> Sent: Tuesday, February 4, 2020 7:57 AM To: Azure/azurehpc <[email protected]> Cc: Mefford, Tim <[email protected]>; Author <[email protected]> Subject: Re: [Azure/azurehpc] Feature request/question on EDR support for FS. (#154) You can actually use the local disks in the HPC VMs but it will probably be a fairly expensive way to create a parallel filesystem (since the disks are only 700GiB per VM). You would just need to make sure you use /dev/sdb when calling the lfsmaster and lfsoss scripts in the install section. This would then allow you to use the IB network. But, if you want compute on that same network you will need to provision more VMs in the scaleset and set the others up as compute (only VMs within a scaleset can communicate with one-another on the IB network). You should only be limited by the network throughput using the Lv2 setup. Can you provide more details about what you are running? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#154?email_source=notifications&email_token=AEMY33OI56EAHKDF5DYQRKLRBGF35A5CNFSM4KN2TBR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKYEOEY#issuecomment-581977875>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AEMY33KY4MM26FMARPGOCXDRBGF35ANCNFSM4KN2TBRQ>.