Pack structs in C bindings about rust-cryptoki HOT 5 OPEN

Well, this is frustrating.

Long story short: Seg faults I was seeing working on #66 are indeed a packing issue. It just took me a while to get around to confirming it.

I now have a clear picture of why following the spec to use 1-byte structure packing results in seg faults: libsofthsm2 doesn't follow it!

When C_GetFunctionList is called, the returned list struct is the 2-byte library version followed by all the function pointers. With default packing (on presumably all 64-bit Linux) this pads the version out to the width of another pointer. This is contrary to the spec and requires an explicit override. However, libsofthsm2 is using default packing everywhere except Windows, returning this (wrong!) platform-default padding. So if a client sets up their structs correctly, as soon as they try to call into a function from the list, they get a value that's part of the correct address and part an adjacent one, seg faulting on deref.

According to this issue, it seems to have become such a deeply ingrained assumption (originating in pk11-kit) it's now a de facto standard they refuse to fix. 🙄

So I guess there's a bit of a philosophical question to be answered here: follow the standard or the mob?

from rust-cryptoki.

Changing the packing for a struct is just a matter of adding an attribute to it at build time, right? Could we add a feature to control that, so that users are free to choose whichever suits them?

from rust-cryptoki.

That's right. However, I don't think there's a way to test the bindings since there's currently only one way to simulate the device. It would require matching compilation(s) of libsofthsm2. There's a preprocessor hook for that, but the current CI setup uses the image default instead of doing the compilation itself.

from rust-cryptoki.

Copying this here from the original PR:

It seems that structure packing should be limited to Windows. This is a known inaccuracy of the spec, and there is a proposal to change this in v3.2. This was accepted without objection according to these meeting minutes.

Even in older versions of the spec, it seems nobody was actually packing structures on Unix. So following the spec literally will cause ABI incompatibility with every existing PKCS#11 implementation.

from rust-cryptoki.

According to this issue, it seems to have become such a deeply ingrained assumption (originating in pk11-kit) it's now a de facto standard they refuse to fix. roll_eyes

AFAIK, it goes back much further than that. See this thread between TC members for some background.

from rust-cryptoki.