RDMA protocol about aws-ofi-nccl HOT 9 OPEN

This absence is specific to Aws-ofi-nccl's RDMA protocol implementation. Context was omitted only for convenience in our first implementation, and can be added in the future.

from aws-ofi-nccl.

Hi @rauteric,

Thank you for the clarification. Our team is considering contributing a fix to introduce FI_CONTEXT support with RDMA and would love to get your thoughts on any implementation preferences or directions your team might have. This would help us tailor our contribution to fit into the project.

from aws-ofi-nccl.

Hi @eliekozah-cornelisnetworks,

Thank you for considering a contribution. In general, I think FI_CONTEXT support can be implemented similarly to how we implement it for the SENDRECV protocol (https://github.com/aws/aws-ofi-nccl/blob/master/src/nccl_ofi_sendrecv.c), by having a fi_context member of the request data structure.

The main obstacle we have currently to supporting FI_CONTEXT is that right now the context we use can be shared between multiple simultaneous Libfabric operations. The particular example I can find is RDMA write operations for individual rails:

There may be other examples of this in the current code, but that's the only one I can find at the moment.

from aws-ofi-nccl.

Hi @rauteric,

I see you removed the check of provider requiring FI_CONTEXT and provider support of RMA here:

from aws-ofi-nccl.

Hi, this was mostly just a refactoring.

The previous code set hints->mode = FI_CONTEXT for both RDMA and SENDRECV protocols (since SENDRECV protocol supports it), so we needed some logic to check that the provider didn't enable it when using RDMA protocol.

Since a recent refactoring, the hints->mode parameter is set separately for the two protocols. The RDMA protocol does not include FI_CONTEXT in hints->mode (which means it does not support it), so it no longer needs the extra check.

Similarly, the check for FI_RMA support from provider is not needed since it is requested as a capability bit in the hints:

from aws-ofi-nccl.