Using file descriptors on Windows? about python-soundfile HOT 5 CLOSED

I would presume that libsndfile is usually compiled using MinGW, whereas Python is compiled with Visual Studio 2006, if I remember correctly. Thus, they use different C runtimes. Apparently, this also means that they use different implementations for file descriptors. File descriptors are opaque pointers as far as C programs are concerned. Each runtime can define them any way they want. This is pretty bad news.

Regardless of whether we can solve this problem by finding a matching set of libsndfile and Python, we can never enforce such a setup for our users. It seems that file descriptors are not a viable option for us. This is why Unix is a good thing...

from python-soundfile.

That's annoying ... did using file descriptors ever work for you on Windows?

It seems that file descriptors are not a viable option for us.

Sadly, it seems so.
Let's discuss possible solutions in #59 and #60.

from python-soundfile.

did using file descriptors ever work for you on Windows?

Within one application, yes. If both the application and libsndfile are linked against the same runtime, everything is fine. This only becomes a problem with dynamic linking and heterogenous C-runtimes. I have wasted too much of my life with the interplay between different compilers and runtimes.

The infamous DLL HELL comes to mind -- if your application links against version X of, say, Qt, and you then dynamically load another library which happens to link against version Y of Qt, all the old Qt symbols in your namespace will get overwritten and your app will crash. This is a common problem with VST plugins.

Similar fun can be had with just installing some software which dumps its libraries to C:\Windows\System32, and overwrites a library that was already there. Library versioning is a relatively new feature in Windows.

To mitigate this problem, Visual Studio mangles all exported symbol names with a version string. This however means that you can't just dynamically link against the DLLs any more, since the function names would be all wrong. The standard practice with Visual Studio is then to link statically against a stub library, which then loads the dynamic library with the correctly mangled names at run time, thus subverting dynamic linking completely.

If you do need to create a DLL with un-mangled names, you need to create a special export-definition file that tells the Visual Studio compiler to use actual C names. Without this, GCC will not link against Visual Studio libraries.

There is also no way of un-loading a linked DLL. This is why Mex-Files sometimes can't be reloaded without restarting Matlab: Once Matlab links the Mex-File, that function is permanent part of Matlab's symbol table and can not be removed without re-starting Matlab (I don't know if this is still true).

And don't even get me started about defective Compilers, strange CPU architectures, or "minimal" embedded runtimes and operating systems. This is why I love the CFFI so much. No compilers, no weird name mangling, no version conflicts, just simple function calls and binary data. Similarly, Python just works on every platform, and is defined abstract enough so that OS differences really don't come into play usually. It is both reassuring and saddening to see problems like these crop up immediately once we leave Python, if only for something as ubiquitous as file descriptors. Maybe it should be a reminder to stick with pure Python, and trust the wisdom of abstracting away all system-level entities.

</rant>

from python-soundfile.

Thanks for the detailed rant. That's interesting. And sad.

from python-soundfile.

I'm closing this because it seems we can't do anything about the problem.

from python-soundfile.