chokkan / simstring Goto Github PK

View Code? Open in Web Editor NEW

113.0 113.0 51.0 158 KB

SimString

Home Page: http://www.chokkan.org/software/simstring/

License: Other

Shell 1.06% C++ 76.14% C 15.06% Java 1.01% Perl 2.40% Python 1.64% Ruby 0.78% Makefile 1.36% HTML 0.56%

simstring's People

Contributors

Stargazers

Watchers

simstring's Issues

Compile pip-installable wheel on Ubuntu 20.10 [solved]

Just in case anyone needs this, this is how I compiled and installed simstring on Ubuntu 20.10. I am sure the pull request #26 is solving the same thing, but I did not notice it until it was too late. So for the record, this "worked for me" (tm)

make sure you have python-dev and swig installed from your package manager

./autogen.sh
./configure
make
cd swig/python
./prepare.sh --swig
pip3 install --upgrade pip setuptools wheel

Now replace setup.py with the version below and then you can:

python3 setup.py bdist_wheel

This gets you a wheel file under dist which you can pip-install as you wish

pip3 install /path/to/simstring/swig/python/dist/simstring.version.whl

Here is a test in python

>>> import simstring
>>> db=simstring.writer("test.db")
>>> db.insert("koira")
>>> db.insert("kissa")
>>> db.insert("autoni")
>>> db.insert("Åbo Akademi")
>>> db.close()
>>> db=simstring.reader("test.db")
>>> db.measure=simstring.cosine
>>> db.threshold=0.5
>>> db.lookup
>>> db.retrieve("Abo akademi")
('Åbo Akademi',)
>>> db.retrieve("auto")
('autoni',)
>>> db.close()

And here is the corrected setup.py:

#!/usr/bin/env python
"""
setup.py file for SWIG example
"""
import sys
import os.path
import setuptools
import sysconfig
def get_rootdir():
    return os.path.abspath(os.path.join(os.path.dirname(__file__), '../..'))
def get_includedir():
    return os.path.join(get_rootdir(), 'include')
def get_swigdir():
    return os.path.join(get_rootdir(), 'swig')
import os; os.environ['CC'] = 'g++'; os.environ['CXX'] = 'g++';
os.environ['CPP'] = 'g++'; os.environ['LDSHARED'] = 'g++'
from setuptools import setup, Extension
simstring_module = Extension(
    '_simstring',
    sources = [
        'export.cpp',
        'export_wrap.cpp',
        ],
    include_dirs=[get_includedir(),],
    extra_link_args=['-shared', sysconfig.get_config_var("BLDLIBRARY")],
    language='c++',
    )
setup(
    name = 'simstring',
    version = '1.1',
    author = 'Naoaki Okazaki',
    description = """SimString Python module""",
    ext_modules = [simstring_module],
    py_modules = ["simstring"],
    )

Python Wrapper Raises TypeError for Unicode Objects

If you pass a Unicode object to SimString the SWIG wrapper will raise a TypeError which to some may be a bit opaque. After reading up on the issues with this I agree with the SWIG design decision not to support Unicode being passed but rather require an explicitly encoded string. I am simply opening this issue (which may of course be closed immediately) so that this has been noted and other users have a point of reference in case they run into similar problems given how Unicode and str objects pretty much can be used interchangeably in Python.

From the SWIG documentation:

At this time, SWIG provides limited support for Unicode and wide-character
strings (the C wchar_t type). Some languages provide typemaps for wchar_t, but
bear in mind these might not be portable across different operating systems.
This is a delicate topic that is poorly understood by many programmers and not
implemented in a consistent manner across languages. For those scripting
languages that provide Unicode support, Unicode strings are often available in
an 8-bit representation such as UTF-8 that can be mapped to the char * type
(in which case the SWIG interface will probably work). If the program you are
wrapping uses Unicode, there is no guarantee that Unicode characters in the
target language will use the same internal representation (e.g., UCS-2 vs.
UCS-4). You may need to write some special conversion functions.

The exception raised:

Traceback (most recent call last):
    File "./simstringutf.py", line <snip>, in <module>
        reader.retrieve(u_str)
    File "<snip>/simstring-1.0/swig/python/simstring.py", line 159, in retrieve
        def retrieve(*args): return _simstring.reader_retrieve(*args)
    TypeError: in method 'reader_retrieve', argument 2 of type 'char const *'

Bourne shell to generate a suitable test database on most GNU/Linux systems:

cat /usr/share/dict/words | simstring --build --unicode --database=words.db

Python code to highlight the issue:

#!/usr/bin/env python
#vim:set encoding=utf-8

from simstring import reader as simstring_reader
from simstring import cosine as simstring_cosine

DB_PATH = 'words.db'

if __name__ == '__main__':
    reader =  simstring_reader(DB_PATH)
    reader.measure = simstring_cosine

    str = '鴨'
    u_str = unicode(str, encoding='utf-8')

    # Will succeed
    print 'Trying to query using a plain string...',
    reader.retrieve(str)
    print 'Done!'

    # Will succeed
    print 'Trying to query using a utf-8 encoded unicode string...',
    reader.retrieve(u_str.encode('utf-8'))
    print 'Done!'

    # Will fail
    print 'Trying to query using a unicode string...',
    reader.retrieve(u_str)
    print 'Done!'

db.treshold has no effect in python

I tested the same database with the command line, where it works and python, where setting the threshold doesn't have any effect. It always seems to be default.

Ruby-binding compilation error in OSX (Mavericks)

I hope that this helps osx users who encountered the same problem.

After I upgraded to Mavericks, I couldn't make the ruby-binding of simstring since running make gave countless error messages such as "undefined templates" and "no matching function."

The problem was the gcc/g++ compilers in Makefile (in my case, it was "CC=clang" and "CXX=clang++").
I installed gcc/g++ through macports and replaced those lines with "CC=gcc" and "CXX=g++"

In addition, compiler makes an error that can't find simstring.h (I am not sure why it can't find it even though extconf.rb has CFLAGS="../../include").
After the include line is fixed (#include <simstring/simstring.h> --> #include <../../include/simstring/simstring.h>), make runs without any problem! :-)

Compilation Error in RedHat linux (gcc version 4.1.2 20080704 (Red Hat 4.1.2-50)

Error messages are as follows,

creating Makefile
compiling export.cpp
In file included from /opt/services/nlp/local/include/simstring/memory_mapped_file.h:62:0,
from /opt/services/nlp/local/include/simstring/simstring.h:52,
from export.cpp:8:
/opt/services/nlp/local/include/simstring/memory_mapped_file_posix.h: In member function ‘void memory_mapped_file_posix::open(const string&, std::ios_base::openmode)’:
/opt/services/nlp/local/include/simstring/memory_mapped_file_posix.h:91:17: error: ‘::close’ has not been declared
::close(m_fd);
^
/opt/services/nlp/local/include/simstring/memory_mapped_file_posix.h: In member function ‘void memory_mapped_file_posix::close()’:
/opt/services/nlp/local/include/simstring/memory_mapped_file_posix.h:106:13: error: ‘::close’ has not been declared
::close(m_fd);
^
/opt/services/nlp/local/include/simstring/memory_mapped_file_posix.h: In member function ‘bool memory_mapped_file_posix::resize(memory_mapped_file_posix::size_type)’:
/opt/services/nlp/local/include/simstring/memory_mapped_file_posix.h:126:17: error: ‘::lseek’ has not been declared
if (::lseek(m_size, size, SEEK_SET) >= 0) {
^
/opt/services/nlp/local/include/simstring/memory_mapped_file_posix.h:128:48: error: ‘read’ was not declared in this scope
if (read(m_fd, &c, sizeof(char)) == -1) {
^
/opt/services/nlp/local/include/simstring/memory_mapped_file_posix.h:131:49: error: ‘write’ was not declared in this scope
if (write(m_fd, &c, sizeof(char)) == -1) {
^
make: *** [export.o] Error 1

I changed export.cpp:8 line from
#include < simstring/simstring.h >
to
#include <../../include/simstring/simstring.h>.

And it works fine.

Cannot build unicode DB on Mac OS X

I cannot use simstring with unicode string.

Followings are what I executed.

$ ./sample_unicode 
libc++abi.dylib: terminate called throwing an exception
Abort trap: 6

It also fails simstring command to build an unicode DB.

$ cat test.txt 
あいうえお
あいうええ
ああああああ
あかさたな
$ simstring -bu -d test.db < test.txt 
libc++abi.dylib: terminate called throwing an exception
Abort trap: 6

Environment:
Mac OS X 10.8.2
XCode 4.6

$ gcc --version
i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)

Please release 1.1

The current master has several build fixes, it would be great if a new release could be made with them. Thanks!

Symbol not found error

I've installed by ruby, but I've got an error below.

LoadError (dlopen(/Users/Macmini/Projects/Pubdictionaries/pubdictionaries/PubDictionaries/lib/simstring/swig/ruby/simstring.bundle, 9): Symbol not found: __ZTIN10__cxxabiv115__forced_unwindE
Referenced from: /Users/Macmini/Projects/Pubdictionaries/pubdictionaries/PubDictionaries/lib/simstring/swig/ruby/simstring.bundle
Expected in: flat namespace
in /Users/Macmini/Projects/Pubdictionaries/pubdictionaries/PubDictionaries/lib/simstring/swig/ruby/simstring.bundle - /Users/Macmini/Projects/Pubdictionaries/pubdictionaries/PubDictionaries/lib/simstring/swig/ruby/simstring.bundle):

Any ideas?

Compiling Yosemite - fails

Same issue mentioned earlier, I just want to point out it also happens on Yosemite.

Fixed by adding the include below to export_wrap.cpp:

#include <stddef.h>

Some compiles not applying duplicate n-gram suffixes

I have encountered a strange bug where my compile of the Python swig wrapper does not add the suffixes for duplicate n-grams from footnote 1 in the paper.

I have been compiling the swig wrapper from a checkout of the repository at d4dca68, using GCC (I believe 9.4.0). I also have a custom Cython wrapper which I've been building against the same checkout of the repository. In some cases the swig wrapper and my wrapper get different output on retrieve() calls. For example, if I insert the single string "assesses" in a database and then retrieve "resssessea", the swig wrapper will match "assesses" but my wrapper will match nothing.

In ngrams() in ngram.h the following code implements the suffixes for duplicate n-grams:

        // Append numbers if the same n-gram occurs more than once.
        for (int i = 2;i <= it->second;++i) {
            stringstream_type ss;
            ss << it->first << i;
            *ins = ss.str();
        }

Adding debugging output either here or in the calling code in simstring.h shows that with my compile of the swig wrapper n-grams all have size 3 even when there are duplicates. My cython wrapper compiled against the same copy of the header file shows two n-grams of size 4 as expected. I don't understand C++ well enough to understand why << i would work in one compile but not another. I tried a number of variations of the code but didn't get anything working with stringstream.

However, the following should be equivalent and does appear to work with both wrappers:

        // Append numbers encoded as sequences of spaces if the same n-gram occurs more than once.
        for (int i = 2;i <= it->second;++i) {
              string_type s;
              s.append(it->first);
              s.append(i - 1, ' ');
              *ins = s;
        }

Build error with Swig > 2.04

Thanks for a great string retrieval package! Been using it with C++ and am setting it up to work with Python.

However, I was unable to build simstring for use with Python using Swig 2.04. My problem relates to the following issues, which were described in the Swig CHANGE doc for version 2.05:

2012-03-20: wsfulton
Fix #3487706 and #3391906 - missing stddef.h include for ptrdiff_t when using %import
for STL containers and compiling with g++-4.6. An include of stddef.h is now only
generated when SWIG generates STL helper templates which require ptrdiff_t. If you
were previously relying on "#include <stddef.h>" always being generated when using a
%include of an STL header, you may now need to add this in manually.

I was able to build simstring with Swig using the method recommended above: specifically, manually placing a "#include <stddef.h>" in the "export_wrap.cpp" file. I don't know Swig well enough to propose a patch which would generate this automatically.

Thanks again.

Using Ruby module: dyld: lazy symbol binding failed: Symbol not found: _libiconv_open

System environment:

OS: OS X Lioon 10.7.5
RVM: rvm 1.22.18
Ruby: ruby 2.0.0p247 (2013-06-27 revision 41674) [x86_64_darwin11]

I tried to use ruby extension of SimString.
It was successfully compiled, but when I ran the sample_unicode.rb (sample.rb runs without any problem) faced the following error messages.

$ ruby sample_unicode.rb

dyld: lazy symbol binding failed: Symbol not found: _libiconv_open
Referenced from: /Users/priancho/work/PubDictionaries/simstring-1.0/swig/ruby/simstring.bundle
Expected in: flat namespace

dyld: Symbol not found: _libiconv_open
Referenced from: /Users/priancho/work/PubDictionaries/simstring-1.0/swig/ruby/simstring.bundle
Expected in: flat namespace

Trace/BPT trap: 5

It seems that the problem comes from two libiconv libraries: one is OS X default library in /usr/lib and the other one is installed by MacPorts in /opt/local/lib.

Googling didn't give much information, but finally one of them worked for me.

In the simstring-1.0/swig/ruby/extconf.rb, put the following line before the create_makefile command as follows.

require 'mkmf'
$CFLAGS='-I../../include'
$LDFLAGS="-lstdc++"

have_library("iconv", "libiconv_open")
create_makefile('simstring')

Then, sample_unicode.rb started work fine.

However, I could not figure out why this one line fixes the problem :-(

simstring python bindings will not build with libc++

Clang on OSX defaults to libc++
The python provided by default in OSX is also linked against libc++, so, adding CPPFLAGS="-stdlib=libstdc++" is not an option.

The problem occurs in ngram.h:99
ngram.h line 99 fails largely because
std::basic_stringstream is only defined for T=char and T=wchar_t in libc++ while simstring tries to initialize it with a bunch of other types.

Is there any way to rewrite this?

Python: export_wrap.cpp: No such file or directory

I get the following error when trying to build the python bindings. I am not sure how to fix it.

$ python setup.py build_ext
running build_ext
building '_simstring' extension
g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -I/home/joseph/utils//include -I/home/joseph/utils/include -fPIC -I/home/joseph/utils/src/simstring.git/include -I/usr/include/python2.6 -c export.cpp -o build/temp.linux-i686-2.6/export.o
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
g++ -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -I/home/joseph/utils//include -I/home/joseph/utils/include -fPIC -I/home/joseph/utils/src/simstring.git/include -I/usr/include/python2.6 -c export_wrap.cpp -o build/temp.linux-i686-2.6/export_wrap.o
g++: export_wrap.cpp: No such file or directory
g++: no input files
error: command 'g++' failed with exit status 1

swig/automake.am has an extra backslash

The following diff is necessary to get autogen.sh to work:

```
  java/Sample.java \
```
```
  java/Sample.java
```

Errors in python/swig on Mac

After the same procedure I followed in Debian build succeeded, Mac fails at extension compilation:

$ python setup.py build_ext
running build_ext
building '_simstring' extension
g++ -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/amadan/simstring/include -I/usr/local/Cellar/python/3.6.5_1/Frameworks/Python.framework/Versions/3.6/include/python3.6m -c export.cpp -o build/temp.macosx-10.13-x86_64-3.6/export.o
In file included from export.cpp:4:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iomanip:48:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/istream:163:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:138:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ios:756:50: error: implicit instantiation of undefined
      template 'std::__1::ctype<unsigned short>'
    return use_facet<ctype<char_type> >(getloc()).widen(__c);
                                                 ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ios:765:19: note: in instantiation of member function
      'std::__1::basic_ios<unsigned short, std::__1::char_traits<unsigned short> >::widen' requested here
        __fill_ = widen(' ');
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:732:39: note: in instantiation of member
      function 'std::__1::basic_ios<unsigned short, std::__1::char_traits<unsigned short> >::fill' requested here
                                 __os.fill()).failed())
                                      ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:1047:19: note: in instantiation of function
      template specialization 'std::__1::__put_character_sequence<unsigned short, std::__1::char_traits<unsigned short> >' requested here
    return _VSTD::__put_character_sequence(__os, __str.data(), __str.size());
                  ^
/Users/amadan/simstring/include/simstring/ngram.h:98:16: note: in instantiation of function template specialization 'std::__1::operator<<<unsigned short,
      std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >' requested here
            ss << it->first << i;
               ^
/Users/amadan/simstring/include/simstring/ngram.h:172:9: note: in instantiation of function template specialization
      'simstring::ngrams<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        ngrams(str, ins, m_n, m_be);
        ^
/Users/amadan/simstring/include/simstring/simstring.h:978:9: note: in instantiation of function template specialization
      'simstring::ngram_generator::operator()<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned
      short> >, std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        gen(query, std::back_inserter(ngrams));
        ^
export.cpp:246:13: note: in instantiation of function template specialization 'simstring::reader::retrieve<simstring::measure::exact,
      std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        dbr.retrieve<simstring::measure::exact>(qstr, threshold, std::back_inserter(xstrs));
            ^
export.cpp:282:9: note: in instantiation of function template specialization 'retrieve_iconv<unsigned short,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<char>, std::__1::allocator<std::__1::basic_string<char> > > > >' requested
      here
        retrieve_iconv<uint16_t>(dbr, query, UTF16, this->measure, this->threshold, std::back_inserter(ret));
        ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:479:52: note: template is declared here
template <class _CharT> class _LIBCPP_TEMPLATE_VIS ctype;
                                                   ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:238:54: error: implicit instantiation of
      undefined template 'std::__1::ctype<unsigned short>'
    return static_cast<const _Facet&>(*__l.use_facet(_Facet::id));
                                                     ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ios:756:12: note: in instantiation of function template
      specialization 'std::__1::use_facet<std::__1::ctype<unsigned short> >' requested here
    return use_facet<ctype<char_type> >(getloc()).widen(__c);
           ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ios:765:19: note: in instantiation of member function
      'std::__1::basic_ios<unsigned short, std::__1::char_traits<unsigned short> >::widen' requested here
        __fill_ = widen(' ');
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:732:39: note: in instantiation of member
      function 'std::__1::basic_ios<unsigned short, std::__1::char_traits<unsigned short> >::fill' requested here
                                 __os.fill()).failed())
                                      ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:1047:19: note: in instantiation of function
      template specialization 'std::__1::__put_character_sequence<unsigned short, std::__1::char_traits<unsigned short> >' requested here
    return _VSTD::__put_character_sequence(__os, __str.data(), __str.size());
                  ^
/Users/amadan/simstring/include/simstring/ngram.h:98:16: note: in instantiation of function template specialization 'std::__1::operator<<<unsigned short,
      std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >' requested here
            ss << it->first << i;
               ^
/Users/amadan/simstring/include/simstring/ngram.h:172:9: note: in instantiation of function template specialization
      'simstring::ngrams<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        ngrams(str, ins, m_n, m_be);
        ^
/Users/amadan/simstring/include/simstring/simstring.h:978:9: note: in instantiation of function template specialization
      'simstring::ngram_generator::operator()<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned
      short> >, std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        gen(query, std::back_inserter(ngrams));
        ^
export.cpp:246:13: note: in instantiation of function template specialization 'simstring::reader::retrieve<simstring::measure::exact,
      std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        dbr.retrieve<simstring::measure::exact>(qstr, threshold, std::back_inserter(xstrs));
            ^
export.cpp:282:9: note: in instantiation of function template specialization 'retrieve_iconv<unsigned short,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<char>, std::__1::allocator<std::__1::basic_string<char> > > > >' requested
      here
        retrieve_iconv<uint16_t>(dbr, query, UTF16, this->measure, this->threshold, std::back_inserter(ret));
        ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:479:52: note: template is declared here
template <class _CharT> class _LIBCPP_TEMPLATE_VIS ctype;
                                                   ^
In file included from export.cpp:4:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iomanip:48:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/istream:163:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:140:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1133:36: error: no matching function for call to
      'use_facet'
    const ctype<_CharT>&    __ct = use_facet<ctype<_CharT> >   (__loc);
                                   ^~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1467:11: note: in instantiation of member
      function 'std::__1::__num_put<unsigned short>::__widen_and_group_int' requested here
    this->__widen_and_group_int(__nar, __np, __ne, __o, __op, __oe, __iob.getloc());
          ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1278:16: note: in instantiation of member
      function 'std::__1::num_put<unsigned short, std::__1::ostreambuf_iterator<unsigned short, std::__1::char_traits<unsigned short> > >::do_put'
      requested here
        return do_put(__s, __iob, __fl, __v);
               ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:462:21: note: in instantiation of member
      function 'std::__1::num_put<unsigned short, std::__1::ostreambuf_iterator<unsigned short, std::__1::char_traits<unsigned short> > >::put' requested
      here
            if (__f.put(*this, *this, this->fill(),
                    ^
/Users/amadan/simstring/include/simstring/ngram.h:98:29: note: in instantiation of member function 'std::__1::basic_ostream<unsigned short,
      std::__1::char_traits<unsigned short> >::operator<<' requested here
            ss << it->first << i;
                            ^
/Users/amadan/simstring/include/simstring/ngram.h:172:9: note: in instantiation of function template specialization
      'simstring::ngrams<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        ngrams(str, ins, m_n, m_be);
        ^
/Users/amadan/simstring/include/simstring/simstring.h:978:9: note: in instantiation of function template specialization
      'simstring::ngram_generator::operator()<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned
      short> >, std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        gen(query, std::back_inserter(ngrams));
        ^
export.cpp:246:13: note: in instantiation of function template specialization 'simstring::reader::retrieve<simstring::measure::exact,
      std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        dbr.retrieve<simstring::measure::exact>(qstr, threshold, std::back_inserter(xstrs));
            ^
export.cpp:282:9: note: in instantiation of function template specialization 'retrieve_iconv<unsigned short,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<char>, std::__1::allocator<std::__1::basic_string<char> > > > >' requested
      here
        retrieve_iconv<uint16_t>(dbr, query, UTF16, this->measure, this->threshold, std::back_inserter(ret));
        ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:236:1: note: candidate template ignored:
      substitution failure [with _Facet = std::__1::ctype<unsigned short>]
use_facet(const locale& __l)
^
In file included from export.cpp:4:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iomanip:48:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/istream:163:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:140:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1135:30: error: implicit instantiation of
      undefined template 'std::__1::numpunct<unsigned short>'
    string __grouping = __npt.grouping();
                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:1421:52: note: template is declared here
template <class _CharT> class _LIBCPP_TEMPLATE_VIS numpunct;
                                                   ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:238:54: error: implicit instantiation of
      undefined template 'std::__1::numpunct<unsigned short>'
    return static_cast<const _Facet&>(*__l.use_facet(_Facet::id));
                                                     ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1134:37: note: in instantiation of function
      template specialization 'std::__1::use_facet<std::__1::numpunct<unsigned short> >' requested here
    const numpunct<_CharT>& __npt = use_facet<numpunct<_CharT> >(__loc);
                                    ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1467:11: note: in instantiation of member
      function 'std::__1::__num_put<unsigned short>::__widen_and_group_int' requested here
    this->__widen_and_group_int(__nar, __np, __ne, __o, __op, __oe, __iob.getloc());
          ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1278:16: note: in instantiation of member
      function 'std::__1::num_put<unsigned short, std::__1::ostreambuf_iterator<unsigned short, std::__1::char_traits<unsigned short> > >::do_put'
      requested here
        return do_put(__s, __iob, __fl, __v);
               ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:462:21: note: in instantiation of member
      function 'std::__1::num_put<unsigned short, std::__1::ostreambuf_iterator<unsigned short, std::__1::char_traits<unsigned short> > >::put' requested
      here
            if (__f.put(*this, *this, this->fill(),
                    ^
/Users/amadan/simstring/include/simstring/ngram.h:98:29: note: in instantiation of member function 'std::__1::basic_ostream<unsigned short,
      std::__1::char_traits<unsigned short> >::operator<<' requested here
            ss << it->first << i;
                            ^
/Users/amadan/simstring/include/simstring/ngram.h:172:9: note: in instantiation of function template specialization
      'simstring::ngrams<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        ngrams(str, ins, m_n, m_be);
        ^
/Users/amadan/simstring/include/simstring/simstring.h:978:9: note: in instantiation of function template specialization
      'simstring::ngram_generator::operator()<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned
      short> >, std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        gen(query, std::back_inserter(ngrams));
        ^
export.cpp:246:13: note: in instantiation of function template specialization 'simstring::reader::retrieve<simstring::measure::exact,
      std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>, std::__1::allocator<unsigned short> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> >, std::__1::allocator<std::__1::basic_string<unsigned short, std::__1::char_traits<unsigned short>,
      std::__1::allocator<unsigned short> > > > > >' requested here
        dbr.retrieve<simstring::measure::exact>(qstr, threshold, std::back_inserter(xstrs));
            ^
export.cpp:282:9: note: in instantiation of function template specialization 'retrieve_iconv<unsigned short,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<char>, std::__1::allocator<std::__1::basic_string<char> > > > >' requested
      here
        retrieve_iconv<uint16_t>(dbr, query, UTF16, this->measure, this->threshold, std::back_inserter(ret));
        ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:1421:52: note: template is declared here
template <class _CharT> class _LIBCPP_TEMPLATE_VIS numpunct;
                                                   ^
In file included from export.cpp:4:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iomanip:48:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/istream:163:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:138:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ios:756:50: error: implicit instantiation of undefined
      template 'std::__1::ctype<unsigned int>'
    return use_facet<ctype<char_type> >(getloc()).widen(__c);
                                                 ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ios:765:19: note: in instantiation of member function
      'std::__1::basic_ios<unsigned int, std::__1::char_traits<unsigned int> >::widen' requested here
        __fill_ = widen(' ');
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:732:39: note: in instantiation of member
      function 'std::__1::basic_ios<unsigned int, std::__1::char_traits<unsigned int> >::fill' requested here
                                 __os.fill()).failed())
                                      ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:1047:19: note: in instantiation of function
      template specialization 'std::__1::__put_character_sequence<unsigned int, std::__1::char_traits<unsigned int> >' requested here
    return _VSTD::__put_character_sequence(__os, __str.data(), __str.size());
                  ^
/Users/amadan/simstring/include/simstring/ngram.h:98:16: note: in instantiation of function template specialization 'std::__1::operator<<<unsigned int,
      std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >' requested here
            ss << it->first << i;
               ^
/Users/amadan/simstring/include/simstring/ngram.h:172:9: note: in instantiation of function template specialization
      'simstring::ngrams<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        ngrams(str, ins, m_n, m_be);
        ^
/Users/amadan/simstring/include/simstring/simstring.h:978:9: note: in instantiation of function template specialization
      'simstring::ngram_generator::operator()<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int>
      >, std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        gen(query, std::back_inserter(ngrams));
        ^
export.cpp:246:13: note: in instantiation of function template specialization 'simstring::reader::retrieve<simstring::measure::exact,
      std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        dbr.retrieve<simstring::measure::exact>(qstr, threshold, std::back_inserter(xstrs));
            ^
export.cpp:285:9: note: in instantiation of function template specialization 'retrieve_iconv<unsigned int,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<char>, std::__1::allocator<std::__1::basic_string<char> > > > >' requested
      here
        retrieve_iconv<uint32_t>(dbr, query, UTF32, this->measure, this->threshold, std::back_inserter(ret));
        ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:479:52: note: template is declared here
template <class _CharT> class _LIBCPP_TEMPLATE_VIS ctype;
                                                   ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:238:54: error: implicit instantiation of
      undefined template 'std::__1::ctype<unsigned int>'
    return static_cast<const _Facet&>(*__l.use_facet(_Facet::id));
                                                     ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ios:756:12: note: in instantiation of function template
      specialization 'std::__1::use_facet<std::__1::ctype<unsigned int> >' requested here
    return use_facet<ctype<char_type> >(getloc()).widen(__c);
           ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ios:765:19: note: in instantiation of member function
      'std::__1::basic_ios<unsigned int, std::__1::char_traits<unsigned int> >::widen' requested here
        __fill_ = widen(' ');
                  ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:732:39: note: in instantiation of member
      function 'std::__1::basic_ios<unsigned int, std::__1::char_traits<unsigned int> >::fill' requested here
                                 __os.fill()).failed())
                                      ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:1047:19: note: in instantiation of function
      template specialization 'std::__1::__put_character_sequence<unsigned int, std::__1::char_traits<unsigned int> >' requested here
    return _VSTD::__put_character_sequence(__os, __str.data(), __str.size());
                  ^
/Users/amadan/simstring/include/simstring/ngram.h:98:16: note: in instantiation of function template specialization 'std::__1::operator<<<unsigned int,
      std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >' requested here
            ss << it->first << i;
               ^
/Users/amadan/simstring/include/simstring/ngram.h:172:9: note: in instantiation of function template specialization
      'simstring::ngrams<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        ngrams(str, ins, m_n, m_be);
        ^
/Users/amadan/simstring/include/simstring/simstring.h:978:9: note: in instantiation of function template specialization
      'simstring::ngram_generator::operator()<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int>
      >, std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        gen(query, std::back_inserter(ngrams));
        ^
export.cpp:246:13: note: in instantiation of function template specialization 'simstring::reader::retrieve<simstring::measure::exact,
      std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        dbr.retrieve<simstring::measure::exact>(qstr, threshold, std::back_inserter(xstrs));
            ^
export.cpp:285:9: note: in instantiation of function template specialization 'retrieve_iconv<unsigned int,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<char>, std::__1::allocator<std::__1::basic_string<char> > > > >' requested
      here
        retrieve_iconv<uint32_t>(dbr, query, UTF32, this->measure, this->threshold, std::back_inserter(ret));
        ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:479:52: note: template is declared here
template <class _CharT> class _LIBCPP_TEMPLATE_VIS ctype;
                                                   ^
In file included from export.cpp:4:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iomanip:48:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/istream:163:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:140:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1133:36: error: no matching function for call to
      'use_facet'
    const ctype<_CharT>&    __ct = use_facet<ctype<_CharT> >   (__loc);
                                   ^~~~~~~~~~~~~~~~~~~~~~~~~
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1467:11: note: in instantiation of member
      function 'std::__1::__num_put<unsigned int>::__widen_and_group_int' requested here
    this->__widen_and_group_int(__nar, __np, __ne, __o, __op, __oe, __iob.getloc());
          ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1278:16: note: in instantiation of member
      function 'std::__1::num_put<unsigned int, std::__1::ostreambuf_iterator<unsigned int, std::__1::char_traits<unsigned int> > >::do_put' requested
      here
        return do_put(__s, __iob, __fl, __v);
               ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:462:21: note: in instantiation of member
      function 'std::__1::num_put<unsigned int, std::__1::ostreambuf_iterator<unsigned int, std::__1::char_traits<unsigned int> > >::put' requested here
            if (__f.put(*this, *this, this->fill(),
                    ^
/Users/amadan/simstring/include/simstring/ngram.h:98:29: note: in instantiation of member function 'std::__1::basic_ostream<unsigned int,
      std::__1::char_traits<unsigned int> >::operator<<' requested here
            ss << it->first << i;
                            ^
/Users/amadan/simstring/include/simstring/ngram.h:172:9: note: in instantiation of function template specialization
      'simstring::ngrams<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        ngrams(str, ins, m_n, m_be);
        ^
/Users/amadan/simstring/include/simstring/simstring.h:978:9: note: in instantiation of function template specialization
      'simstring::ngram_generator::operator()<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int>
      >, std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        gen(query, std::back_inserter(ngrams));
        ^
export.cpp:246:13: note: in instantiation of function template specialization 'simstring::reader::retrieve<simstring::measure::exact,
      std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        dbr.retrieve<simstring::measure::exact>(qstr, threshold, std::back_inserter(xstrs));
            ^
export.cpp:285:9: note: in instantiation of function template specialization 'retrieve_iconv<unsigned int,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<char>, std::__1::allocator<std::__1::basic_string<char> > > > >' requested
      here
        retrieve_iconv<uint32_t>(dbr, query, UTF32, this->measure, this->threshold, std::back_inserter(ret));
        ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:236:1: note: candidate template ignored:
      substitution failure [with _Facet = std::__1::ctype<unsigned int>]
use_facet(const locale& __l)
^
In file included from export.cpp:4:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iomanip:48:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/istream:163:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:140:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1135:30: error: implicit instantiation of
      undefined template 'std::__1::numpunct<unsigned int>'
    string __grouping = __npt.grouping();
                             ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:1421:52: note: template is declared here
template <class _CharT> class _LIBCPP_TEMPLATE_VIS numpunct;
                                                   ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:238:54: error: implicit instantiation of
      undefined template 'std::__1::numpunct<unsigned int>'
    return static_cast<const _Facet&>(*__l.use_facet(_Facet::id));
                                                     ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1134:37: note: in instantiation of function
      template specialization 'std::__1::use_facet<std::__1::numpunct<unsigned int> >' requested here
    const numpunct<_CharT>& __npt = use_facet<numpunct<_CharT> >(__loc);
                                    ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1467:11: note: in instantiation of member
      function 'std::__1::__num_put<unsigned int>::__widen_and_group_int' requested here
    this->__widen_and_group_int(__nar, __np, __ne, __o, __op, __oe, __iob.getloc());
          ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/locale:1278:16: note: in instantiation of member
      function 'std::__1::num_put<unsigned int, std::__1::ostreambuf_iterator<unsigned int, std::__1::char_traits<unsigned int> > >::do_put' requested
      here
        return do_put(__s, __iob, __fl, __v);
               ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/ostream:462:21: note: in instantiation of member
      function 'std::__1::num_put<unsigned int, std::__1::ostreambuf_iterator<unsigned int, std::__1::char_traits<unsigned int> > >::put' requested here
            if (__f.put(*this, *this, this->fill(),
                    ^
/Users/amadan/simstring/include/simstring/ngram.h:98:29: note: in instantiation of member function 'std::__1::basic_ostream<unsigned int,
      std::__1::char_traits<unsigned int> >::operator<<' requested here
            ss << it->first << i;
                            ^
/Users/amadan/simstring/include/simstring/ngram.h:172:9: note: in instantiation of function template specialization
      'simstring::ngrams<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        ngrams(str, ins, m_n, m_be);
        ^
/Users/amadan/simstring/include/simstring/simstring.h:978:9: note: in instantiation of function template specialization
      'simstring::ngram_generator::operator()<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int>
      >, std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        gen(query, std::back_inserter(ngrams));
        ^
export.cpp:246:13: note: in instantiation of function template specialization 'simstring::reader::retrieve<simstring::measure::exact,
      std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>, std::__1::allocator<unsigned int> >,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> >, std::__1::allocator<std::__1::basic_string<unsigned int, std::__1::char_traits<unsigned int>,
      std::__1::allocator<unsigned int> > > > > >' requested here
        dbr.retrieve<simstring::measure::exact>(qstr, threshold, std::back_inserter(xstrs));
            ^
export.cpp:285:9: note: in instantiation of function template specialization 'retrieve_iconv<unsigned int,
      std::__1::back_insert_iterator<std::__1::vector<std::__1::basic_string<char>, std::__1::allocator<std::__1::basic_string<char> > > > >' requested
      here
        retrieve_iconv<uint32_t>(dbr, query, UTF32, this->measure, this->threshold, std::back_inserter(ret));
        ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/__locale:1421:52: note: template is declared here
template <class _CharT> class _LIBCPP_TEMPLATE_VIS numpunct;
                                                   ^
10 errors generated.
error: command 'g++' failed with exit status 1
(venv) 909 (master) python@ddhcp219125$

Swig: 3.0.12
G++: 4.2.1 - Apple LLVM version 10.0.0 (clang-1000.11.45.5)

Compilation error on LinuxUbuntu

Hi ,
I follow instructions and running ./configure and then make - which fails with following error:
Any chance you can help to figure out this quickly ?

make[2]: Entering directory /home/tserver/Public/brat-v1.3_Crunchy_Frog/simstring-1.0/include' make[2]: Nothing to be done forall'.
make[2]: Leaving directory /home/tserver/Public/brat-v1.3_Crunchy_Frog/simstring-1.0/include' Making all in frontend make[2]: Entering directory/home/tserver/Public/brat-v1.3_Crunchy_Frog/simstring-1.0/frontend'
g++ -DHAVE_CONFIG_H -I. -I.. -I.. -I../include -O3 -ffast-math -O3 -ffast-math -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.cpp
In file included from ../include/simstring/memory_mapped_file.h:62:0,
from ../include/simstring/simstring.h:52,
from main.cpp:43:
../include/simstring/memory_mapped_file_posix.h: In member function âvoid memory_mapped_file_posix::open(const string&, std::ios_base::openmode)â:
../include/simstring/memory_mapped_file_posix.h:91:17: error: â::closeâ has not been declared
../include/simstring/memory_mapped_file_posix.h: In member function âvoid memory_mapped_file_posix::close()â:
../include/simstring/memory_mapped_file_posix.h:106:13: error: â::closeâ has not been declared
../include/simstring/memory_mapped_file_posix.h: In member function âbool memory_mapped_file_posix::resize(memory_mapped_file_posix::size_type)â:
../include/simstring/memory_mapped_file_posix.h:126:17: error: â::lseekâ has not been declared
../include/simstring/memory_mapped_file_posix.h:128:48: error: âreadâ was not declared in this scope
../include/simstring/memory_mapped_file_posix.h:131:49: error: âwriteâ was not declared in this scope
make[2]: *** [main.o] Error 1
make[2]: Leaving directory /home/tserver/Public/brat-v1.3_Crunchy_Frog/simstring-1.0/frontend' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory/home/tserver/Public/brat-v1.3_Crunchy_Frog/simstring-1.0'
make: *** [all] Error 2

Documentation on simstring.writer ?

In the Python code same, you do:

Open a SimString database for writing with Unicode mode.

db = simstring.writer('sample_unicode.db', 3, False, True)

Where can I find the documentation for the last three parameters?
Where are they documented?

Unable to create PHP extension with swig

Hi,

I was working on a PHP extension using Swig, but I'm a little stuck right now.

I followed the Swig PHP documentation but I got an error.

Here is what I have done and the result I got, maybe you can help me with that:

cd swig;
mkdir php;
# I will make a 'prepare.sh' later, but this is for the example
ln -s ../export.cpp
ln -s ../export.h
ln -s ../export.i
swig -c++ -php -o export_wrap.cpp export.i
g++ `php-config --includes` -fPIC -c export_wrap.cpp
g++ -shared export_wrap.o -o simstring.so
sudo mv simstring.so my-php-extension-dir/

I load the extension in my php.ini but I got the following error :

PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib/php5/20121212/simstring.so' - /usr/lib/php5/20121212/simstring.so: undefined symbol: _ZN6writer5closeEv in Unknown on line 0

Thank you if you have a hint on that one !

I will make a PR if I manage to make this work.

cannot make-cc1plus warnings lead to error

Hi dears,
In system:
Fedora 21
Ruby 2.1.7
GCC 4.9.2

I follow the instructions but afer
$./prepare.sh
$ruby extconf.rb
when trying to $make and $make install, I got an error:

cc1plus: some warnings being treated as errors
Makefile:217: recipe for target 'export_wrap.o' failed
make: *** [export_wrap.o] Error 1

And the entire log of this error is bellow
cc1plus error.txt

I searched about this error but all of replies are about how to disable Werrors and etc.:-(
Is there any solution?

Sorting based on similarity

In there anyway to retrieve the most similar string or sorted all the retrieved ones based on the similarity score?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.