Questions about Non-Coherent memory and nonCoherentAtomSize about vulkanmemoryallocator HOT 4 CLOSED

First of all, thank you for your question and deep analysis of the problem. Discussions like this help to improve quality of the library.

Regarding your first question: Yes, this is correct. vmaMapMemory is synchronized internally in the way that it is safe to call it on different VmaAllocation objects that have the same VkDeviceMemory from different threads.

Regarding your last question: No, setting preferredLargeHeapBlockSize doesn't mean such block size is guaranteed. Some blocks may be allocated as smaller or larger, depending on internal library heuristics. More specifically, first blocks are smaller not to allocate such large block if you actually use only little memory from certain type.

That said, I must admit that the library here has some room for improvement in the subject discussed here. Please feel free to propose improvements to the library that you would like to see. I'm willing to work on this and release a new version that will support non-coherent memory better. First ideas that come to my mind are:

1. For each memory type that is HOST_VISIBLE but not HOST_COHERENT, allocations would be automatically aligned to nonCoherentAtomSize and always lie on separate pages of this size. Block size would also always be multiple of nonCoherentAtomSize.

2. Convenience functions could be added to support Flush and Invalidate, like:

void vmaFlushAllocation(VmaAllocation allocation, VkDeviceSize offset, VkDeviceSize size);
void vmaInvalidateAllocation(VmaAllocation allocation, VkDeviceSize offset, VkDeviceSize size);

In the meantime, you can just #define VMA_DEBUG_ALIGNMENT 256. If your preferredLargeHeapBlockSize is a multiply of 2 MB (by default it is), you can be sure that size of allocated block is also a multiply of 256 (as long as it's not a dedicated allocation) - which is the largest nonCoherentAtomSize found in the wild now, according to http://vulkan.gpuinfo.org/listlimits.php. You should be safe then.

from vulkanmemoryallocator.

Thanks for the response and the suggestion for the current way to workaround the issue.

For our current library we went with an approach which is basically like option 1. that you suggested above. It's the easier one to implement as you just need to update the alignment when you're allocating the block. However, it does mean you end up with less flexibility in how you sub-allocate and you can end up with more wasted space.

Option 2 is nice in that it shouldn't effect the allocation/sub-allocation schemes at all, and works as long as the offset and size passed in are within the individual VmaAllocation (easy enough to assert/check). The library should then have enough knowledge to know how to pad the beginning and ends to make sure it is aligned with nonCoherentAtomSize. This approach does require that you are always mapping the entire the VkDeviceMemory incase the padding enters into other VmaAllocations. This is what the library currently does so it should be fine, but if there is any future plans to not map the entire VkDeviceMemory this will then have to be taken into consideration.

Option2 does mean that the library will need to do some additional work each time flush or invalidate is called to calculate the padding. However, this is work that the client would/should be doing otherwise so it shouldn't add any more work in total, excluding the scenario where we used option 1 and only sub-allocations got mapped and the client always flushed/invalidated the entire mapping (thus no need for checking alignments).

Anyways given the current library, I think option 2 would be the better, most flexible approach.

from vulkanmemoryallocator.

I didn't mean either 1 or 2 as alternative options, but rather both. I'm sorry if that wasn't clear enough.

I think that aligning allocations to nonCoherentAtomSize always is right thing to do. Even the worst case of 256B it's still just 16*vec4. I think that such alignment wouldn't result in too much wasted space, as using large number of buffers smaller than that doesn't sound like a good practice anyway.

As you noticed, the library always maps entire VkDeviceMemory object. There are no plans to change that.

from vulkanmemoryallocator.

@egdaniel I'm sorry for the delay. I pushed major update with support for non-coherent memory to "development" branch, as I described above. Can you please check if it works and looks good to you?

from vulkanmemoryallocator.