Need binaries for libsndfile 1.0.26 about python-soundfile HOT 22 OPEN

Hi.

It seems to me that the chunk API is not finished yet. There's no documentation on the website yet, and 1.0.26 isn't mentioned in the news or changelog on the website. Also, chunk information is not available for all file formats and all chunk types.

The more important issue is that I want PySoundFile to be a library for reading/writing sound files in Python, and not necessarily a complete set of bindings for libsndfile. To my mind, libsndfile is an implementation detail that should be invisible to users. To that end, I don't think I want to support the chunk API in PySoundFile directly.

However, I was unaware that there is a new version of libsndfile, and will update the binaries and packages accordingly. You can also do this locally if you don't want to wait for me. You can then use your code example in your programs to use the chunk API by subclassing SoundFile and adding new methods. You will need to add some cdefs and re-dlopen the library for this to work, though.

from python-soundfile.

I have to rebuild for linux anyway it seems because libsndfile is not up to date with either ubuntu or debian.

I have subclassed it, and worked with them to figure out the new commands. It looks like this to check and get an ixml chunk.

def get_ixml(self):

        chunk_info = _ffi.new("SF_CHUNK_INFO*")
        chunk_info.id = b'iXML'
        chunk_info.id_size = 4;

        iterator = _snd.sf_get_chunk_iterator(self._file, chunk_info)

        err = _snd.sf_get_chunk_size(iterator, chunk_info)

        if (chunk_info.datalen > 0):
            chunk_info.data = _snd.malloc (chunk_info.datalen) ;
            _snd.sf_get_chunk_data(iterator, chunk_info)
            self.ixml_info = _ffi.buffer(chunk_info.data, chunk_info.datalen)[:].decode()
            _snd.free(chunk_info.data)

Note this

chunk_info.data = _snd.malloc (chunk_info.datalen) ;

I'm not across cffi much to figure out how to allocate memory to a struct yet. The "new" command wasn't working for me and was segfaulting. I've asked cffi group how to do this.

The cdef looks like this

typedef struct SF_BROADCAST_INFO
{
                char        description [256];
                char        originator [32] ;
                char        originator_reference [32] ;
                char        origination_date [10] ;
                char        origination_time [8] ;
                uint32_t    time_reference_low ;
                uint32_t    time_reference_high ;
                short       version ;
                char        umid [64] ;
                char        reserved [190] ;
                uint32_t    coding_history_size ;
                char        coding_history [2048] ;
} SF_BROADCAST_INFO ;

struct SF_CHUNK_INFO
{   char        id [64] ;   /* The chunk identifier. */
    unsigned    id_size ;   /* The size of the chunk identifier. */
    unsigned    datalen ;   /* The size of that data. */
    void        *data ;     /* Pointer to the data. */
} ;

And for broadcast wave data

def get_bext(self):
        """Retrieve the log string generated when opening the file."""
        info = _ffi.new("SF_BROADCAST_INFO*")
        _snd.sf_command(self._file, _snd.SFC_GET_BROADCAST_INFO,
                        info, 1024)

        self.bext_info = {
            "description": self.ffi_string(info.description),
            "originator": self.ffi_string(info.originator),
            "originator_reference": self.ffi_string(info.originator_reference),
            "origination_date":  self.ffi_string(info.origination_date),
            "origination_time":  self.ffi_string(info.origination_time),
            "timereference_translated": samples_to_time(info.time_reference_low, self.samplerate),
            "timereference": info.time_reference_low,
            "version": info.version,
            "umid": self.ffi_string(info.umid),
            "coding_history": self.ffi_string(info.coding_history)


        }

If you think any of this is useful I can work something out.

from python-soundfile.

I've released the extension for extracting the bext and iXML chunks after getting info on allocating memory to the struct without using malloc. It seems you need to hold the pointer in a variable to prevent segfaulting. I believe it will be cleared once the program ends, but not sure about in a loop iterating through files ?

It just requires the updated libsndfile version. I've yet to check what happens when the library is called within a virtualenv.

https://github.com/danrossi/bwfsoundlib

from python-soundfile.

This is very cool! I'll put a link to it in the README!

from python-soundfile.

Would that be more useful integrated at all or best to separate the bloat ? Those extra defs might work best in the main project perhaps to help with documenting the interface ?

due to the fact it requires a specific version. How would I go building an installer then with the specific binaries ? How does your package work installing the binaries ? Should I make a pull request for the updated binaries ?

What about for linux, I guess I need to contact both debian and ubuntu ? They only have 1.0.25.

Ive tested this processing a directory of files. I'm running python 3.5 to get the fast efficient scandir function and with this it processes the files and extracts information I need to index to a database in seconds. I still need to test for any memory leaks I hope they get cleaned up within an iteration.

from python-soundfile.

I've made a request that the ubuntu package get updated but there might be a need to still deliver the binary directly ?

I need it specifically to work on embedded NAS systems like the synology and it seems if its the same arch , the ubuntu binary will run on it.

https://answers.launchpad.net/ubuntu/+source/libsndfile/+question/280926

from python-soundfile.

I could put a binary into the wheel for Linux, but I worry that a single binary would not work across distributions, since libraries are in different places etc. If you want to patch your current installation, though, just put a more up-to-date libsndfile.a in your working directory.

PySoundFile tries to load a local libsndfile before trying to load the wheel-provided one. If you have a libsndfile in your current directory, it should get loaded instead of the system-installed one.

As for installers, I recommend building pip wheels instead. You could bundle your custom libsndfile with your own bwfsoundlib.

Be warned, though. Bundling binaries with a pip wheel is a terrifying pain. Have a look at class bdist_wheel_half_pure in setup.py to see all the gory details. Basically, you have to create a Python-independant, but OS-dependant commandclass, and then add your binary as an additional package. It's not pretty, but it works.

from python-soundfile.

OK thanks for that no worries. If you need me to create a pull request for the OSX and windows binaries I can have a go compiling them up. I'm on OSX and actually did have to compile the new version and copied over the lib to that path you mentioned manually. Could the same be done for the linux library also ?

from python-soundfile.

If you would do the compiling, I'd be grateful!

I am still doubtful whether the linux library would work across distributions. If you compile one, I could try in on a few different distributions, though.

from python-soundfile.

OK. I have forked that rep and will try and make a Makefile out of it also. I have made a request with ubuntu to have the package updated here

https://bugs.launchpad.net/ubuntu/+source/libsndfile/+bug/1534098

I use macports on OSX so perhaps it could be installed via a homebrew and ports package also ?

from python-soundfile.

It seems that macports and homebrew is up to date. Perhaps the makefile could check for that and install and copy that over instead.

https://trac.macports.org/browser/trunk/dports/audio/libsndfile/Portfile

from python-soundfile.

I don't think it should copy the system binary. Theoretically, pysoundfile should prefer the system binary over the wheel one anyway.

from python-soundfile.

Let me add my 2 cents ...

Linux

I don't think we should provide binaries for Linux. For normal Linux users it should be no problem to locally compile whatever libsndfile version they want. And as long as it's a meaningful location (/usr/local/lib or similar), PySoundFile will automatically use the locally compiled library. For those Linux users who don't know how to compile stuff, it's just a matter of time until the new libsndfile version can be installed with the package manager.
For the Debian package see https://tracker.debian.org/pkg/libsndfile/. It looks like @erikd himself is the maintainer, so I'm wondering why it wasn't updated yet ...

Windows

Shortly after libsndfile 1.0.26 was announced, I compiled DLLs for it: https://github.com/bastibe/libsndfile-binaries/tree/update-to-1.0.26
Sadly, the new DLL (I only tried the 64bit one) breaks 32 of our unit tests, mainly because perfectly normal files are suddenly reported as not seekable. Here's an example how to reproduce this:

>>> import soundfile as sf
>>> f = sf.SoundFile('test.wav')
>>> f.seekable()
False

Here's the full test output: error.txt.

I also tried it with the DLLs published by @erikd (http://www.mega-nerd.com/libsndfile/files/libsndfile-1.0.26-w64-setup.exe) and they show the same erroneous behavior.

@erikd: Is this a known error?

I currently don't have the stamina to install a compiler on Windows and compile a test program in C using libsndfile directly, can somebody else please try that?

Mac OS X

I didn't try that yet, instructions for compilation are there: https://github.com/bastibe/libsndfile-binaries#dylib-for-mac-os-x-64-bit.
I can probably try this in the next few days ...

from python-soundfile.

That seekable bug actually explains some weird behavior I have seen lately.

from python-soundfile.

Are you saying this version produces a regression ? I have not run unit tests on it. I have compiled it on osx fine and it runs fine for my needs.

I have requested ubuntu update it but if there is a regression that is a problem.

My end goal is to have to packaged up in installers to compiling it for linux might still be ok. A makefile could install it on osx using macports or homebrew if available also.

from python-soundfile.

I'm not sure. I have seen strange behavior lately, but I haven't gotten to the bottom of it yet. I hope that I'll have time next week or so to look into it.

from python-soundfile.

There was a seek flag bug that was introduced in commit libsndfile/libsndfile@19e12f564 and ended up in the 1.0.26 release.

It was fixed in these two comits:

libsndfile/libsndfile@776308d
libsndfile/libsndfile@f29c0dc

and will be in the upcoming 1.0.27 release.

from python-soundfile.

OK. sounds significant to hold out then.

from python-soundfile.

Thanks @erikd for the clarification (and for fixing this stuff)!

I initially thought this is a Windows problem, but apparently it affects all OSs.

I'm looking forward to the 1.0.27 release!

from python-soundfile.

Thank you @erikd, this does indeed explain the strange behavior I was seeing (openSuse had 1.0.26 installed already). Let's wait for 1.0.27 with the new binaries for PySoundFile.

from python-soundfile.

http://www.mega-nerd.com/libsndfile/ announced Version 1.0.27 (June 19 2016).

from python-soundfile.

I updated to 1.0.27 in #171.

from python-soundfile.