Better Finnish analyzer based on libvoikko about annif HOT 3 CLOSED

There's a new voikko module on PyPI: https://pypi.org/project/voikko/#description
Maybe it would help with the install problems?

from annif.

There is an initial implementation on the branch issue37-voikko.

However, there are (at least) two outstanding issues:

• The Travis build fails because using the voikko backend would require installing the libvoikko library from a deb package. Currently we are using the container based infrastructure, which limits package installs to whitelisted packages, and libvoikko is not on the whitelist. More generally, do we want to add a feature that requires installing a system library? At least it shouldn't be a hard dependency - basic functionality and unit tests should work even without installing libvoikko.
• There are weird ValueError: ctypes objects containing pointers cannot be pickled in unit tests. I don't understand what triggers these, as the voikko analyzer shouldn't be involved in what is being pickled, it just inputs and outputs normal Python strings.

from annif.

Travis has deprecated their container environment and we have switched to the Xenial based environment where installing any deb package is possible. So it should be possible to run the tests with Voikko on Travis. However, the voikko feature should be made optional so that native libraries are not required for a basic install of Annif. (See also #229 for similar thoughts about fastText)

The pickle issue has been sorted out on the issue37-voikko branch.

from annif.