languagemachines / timbl Goto Github PK

TiMBL implements several memory-based learning algorithms.

Home Page: https://languagemachines.github.io/timbl

License: GNU General Public License v3.0

Shell 0.76% C++ 97.87% Makefile 0.28% M4 0.82% Dockerfile 0.26%

machine-learning classification learning-algorithm timbl ib1 igtree decision-tree k-nearest-neighbours nearest-neighbours knn

timbl's Introduction

=========================================== TiMBL: Tilburg Memory Based Learner

TiMBL 6.4 (c) CLS/ILK/CLiPS 1998 - 2024
Centre for Language Studies, Radboud University Nijmegen
Induction of Linguistic Knowledge Research Group, Tilburg University and
Centre for Dutch Language and Speech, University of Antwerp

Website: https://languagemachines.github.io/timbl/

TiMBL is an open source software package implementing several memory-based learning algorithms, among which IB1-IG, an implementation of k-nearest neighbor classification with feature weighting suitable for symbolic feature spaces, and IGTree, a decision-tree approximation of IB1-IG. All implemented algorithms have in common that they store some representation of the training set explicitly in memory. During testing, new cases are classified by extrapolation from the most similar stored cases.

For over fifteen years TiMBL has been mostly used in natural language processing as a machine learning classifier component, but its use extends to virtually any supervised machine learning domain. Due to its particular decision-tree-based implementation, TiMBL is in many cases far more efficient in classification than a standard k-nearest neighbor algorithm would be.

This is a major extension to the sixth main release of TiMBL. Most significant change: The main program is now called 'timbl' and not 'Timbl' anymore. Be warned! This change is part of our effort to get our MBL software into software distributions like Debian, Ubuntu, RedHat .

Comments and bug-reports are welcome at our issue tracker at https://github.com/LanguageMachines/timbl/issues or by mailing lamasoftware (at) science.ru.nl. Documentation and more info may be found on https://languagemachines.github.io/timbl .

TiMBL is distributed under the GNU Public Licence v3 (see the file COPYING).

This software has been tested on:

Intel platforms running several versions of Linux, including Ubuntu, Debian, Arch Linux, Fedora (both 32 and 64 bits)
MAC platform running OS X 10.10

Alternatively, with some effort, you may get it to work on a Windows platform using Cygwin.

Compilers:

GCC (use 7.0 or later)
Clang

Contents of this distribution:

Sources
Licensing information ( COPYING )
Build system based on GNU Autotools
Container build file ( Dockerfile )
Example data files ( in the demos directory )
Documentation ( in the docs directory )

Dependencies: To be able to succesfully build TiMBL from the tarball, you need the following pakages:

ticcutils (https://github.com/LanguageMachines/ticcutils)
pkg-config
libxml2-dev

To install TiMBL, first consult whether your distribution's package manager has an up-to-date package for TiMBL.

To compile and install manually from source instead, provided you have all the dependencies installed:

$ bash bootstrap.sh
$ ./configure
$ make
$ make install

If you want to automatically download and install the latest stable versions of the required dependencies, then run ./build-deps.sh prior to the above. You can pass a target directory prefix as first argument and you may need to prepend sudo to ensure you can install there. The dependencies are:

ticcutils

A Dockerfile for a container build is also available, specify --build-arg VERSION=development if you want the latest development version instead.

You will still need to take care to install the following 3rd party dependencies through your distribution's package manager, as they are not provided by our script:

icu - A C++ library for Unicode and Globalization support. On Debian/Ubuntu systems, install the package libicu-dev.
A sane build environment with a C++ compiler (e.g. gcc 4.9 or above or clang), make, autotools, libtool, pkg-config

timbl's People

Contributors

Stargazers

Watchers

Forkers

anukat2015 kangdazhi juicedd yetianlinguistics berhan-h n8wacht vivekanon sanyaade-projects fellowtraveler

timbl's Issues

[lintian warnings] manpage-has-errors-from-man

W: timbl: manpage-has-errors-from-man usr/share/man/man1/timbl.1.gz 305: warning: numeric expression expected (got `o')                                                                        
N:                                                                                                                                                                                             
N:    This man page provokes warnings or errors from man.                                                                                                                                      
N:                                                                                                                                                                                             
N:    "cannot adjust" or "can't break" are trouble with paragraph filling,                                                                                                                     
N:    usually related to long lines. Adjustment can be helped by left
N:    justifying, breaks can be helped with hyphenation, see "Manipulating
N:    Filling and Adjusting" and "Manipulating Hyphenation" in the groff
N:    manual (see info groff).                                                                                     
N:                
N:    "can't find numbered character" usually means latin1 etc in the input,
N:    and this warning indicates characters will be missing from the output.
N:    You can change to escapes like \[:a] described on the groff_char man
N:    page.                              
N:
N:    Other warnings are often formatting typos, like missing quotes around a
N:    string argument to .IP. These are likely to result in lost or malformed
N:    output. See the groff_man (or groff_mdoc if using mdoc) man page for
N:    information on macros.                                                 
N:                                                                       
N:    This test uses man's --warnings option to enable groff warnings that
N:    catch common mistakes, such as putting . or ' characters at the start of
N:    a line when they are intended as literal text rather than groff   
N:    commands. This can be fixed either by reformatting the paragraph so that
N:    these characters are not at the start of a line, or by adding a
N:    zero-width space (\&) immediately before them.
N:
N:    At worst, warning messages can be disabled with the .warn directive, see
N:    "Debugging" in the groff manual.
N:
N:    Lintian also stricter in regards to declaring manpage preprocessors.
N:
N:    To test this for yourself you can use the following command:
N:     LC_ALL=en_US.UTF-8 MANROFFSEQ='' MANWIDTH=80 \
N:            man --warnings -E UTF-8 -l -Tutf8 -Z <file> >/dev/null
N:
N:    Refer to the groff_man(7) manual page and the groff_mdoc(7) manual page
N:    for details.
N:
N:    Severity: normal, Certainty: certain

also:

I: timbl: spelling-error-in-manpage usr/share/man/man1/timbl.1.gz distibutions distributions
I: timbl: spelling-error-in-manpage usr/share/man/man1/timbl.1.gz SYNOPSYS SYNOPSIS
I: libtimbl4: spelling-error-in-binary usr/lib/libtimbl.so.4.0.1 occurence occurrence
I: libtimbl4: spelling-error-in-binary usr/lib/libtimbl.so.4.0.1 erronous erroneous

installation problems

Hi,
After installing ticcutils successfully, I met some problems in installing timbl.
It seems that I have installed all dependencies listed; however, after implementing "make", I have got some errors like the following:
make[2]: Entering directory '/home/kun/Software/entropy/timbl-6.4.12/demos'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/kun/Software/entropy/timbl-6.4.12/demos'
Making all in docs
make[2]: Entering directory '/home/kun/Software/entropy/timbl-6.4.12/docs'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/kun/Software/entropy/timbl-6.4.12/docs'
Making all in m4
make[2]: Entering directory '/home/kun/Software/entropy/timbl-6.4.12/m4'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/home/kun/Software/entropy/timbl-6.4.12/m4'
make[2]: Entering directory '/home/kun/Software/entropy/timbl-6.4.12'
make[2]: Leaving directory '/home/kun/Software/entropy/timbl-6.4.12'
make[1]: Leaving directory '/home/kun/Software/entropy/timbl-6.4.12'
% make install
Making install in src
make[1]: Entering directory '/home/kun/Software/entropy/timbl-6.4.12/src'
make[2]: Entering directory '/home/kun/Software/entropy/timbl-6.4.12/src'
/bin/mkdir -p '/usr/local/lib'
/bin/bash ../libtool --mode=install /usr/bin/install -c libtimbl.la '/usr/local/lib'
libtool: install: /usr/bin/install -c .libs/libtimbl.so.4.0.1 /usr/local/lib/libtimbl.so.4.0.1
/usr/bin/install: cannot remove '/usr/local/lib/libtimbl.so.4.0.1': Permission denied
Makefile:584: recipe for target 'install-libLTLIBRARIES' failed
make[2]: *** [install-libLTLIBRARIES] Error 1
make[2]: Leaving directory '/home/kun/Software/entropy/timbl-6.4.12/src'
Makefile:1006: recipe for target 'install-am' failed
make[1]: *** [install-am] Error 2
make[1]: Leaving directory '/home/kun/Software/entropy/timbl-6.4.12/src'
Makefile:465: recipe for target 'install-recursive' failed

Who would like to help me solve the problem.

Error while compiling timbl-6.4.12

On CentOS 7. Getting the following error...

/bin/sh ../libtool --tag=CXX --mode=link g++ -std=c++11 -W -Wall -O3 -g -pedantic -g -O2 -fopenmp -I/usr/include/libxml2 -fopenmp -I/usr/local/include -o timbl Timbl.o libtimbl.la -lxml2 -L/usr/local/lib -lticcutils
libtool: link: g++ -std=c++11 -W -Wall -O3 -g -pedantic -g -O2 -fopenmp -I/usr/include/libxml2 -fopenmp -I/usr/local/include -o .libs/timbl Timbl.o ./.libs/libtimbl.so -lxml2 -L/usr/local/lib /usr/local/lib/libticcutils.so -pthread -fopenmp -Wl,-rpath -Wl,/usr/local/lib
/usr/local/lib/libticcutils.so: undefined reference to boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > >::maybe_assign(boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > > const&)' /usr/local/lib/libticcutils.so: undefined reference to boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > >, boost::regex_traits<char, boost::cpp_regex_traits > >::construct_init(boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits > > const&, boost::regex_constants::_match_flags)'
collect2: error: ld returned 1 exit status
make[2]: *** [timbl] Error 1
make[2]: Leaving directory /tmp/timbl-6.4.11/src' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory /tmp/timbl-6.4.11'
make: *** [all] Error 2

Any ideas on how to resolve this?

Thanks!
Lou

No proper exit code after error

I get the error reading an instancebase(-i option) without a testfile (-t option) is useless (and rightly so), but timbl exits with 0 instead of a proper error code... (and therefore LuigiNLP doesn't realize it failed)

add a parameter to select the N best features

For some tasks, with a lot of features, it might be handy to let Timbl select the top N features, based on the current weight, and use only those to build the tree. Also an implicit -mI for all the other features.

e.g a --cutoff 1000, would select the 1000 'best ranked' features. Assuming more than 1000 are available :)

comment welcome....

/usr/local/lib/libticcutils.so: undefined reference to `boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > > >::maybe_assign(boost::match_results<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > > > const&)'
/usr/local/lib/libticcutils.so: undefined reference to `boost::re_detail::perl_matcher<__gnu_cxx::__normal_iterator<char const*, std::string>, std::allocator<boost::sub_match<__gnu_cxx::__normal_iterator<char const*, std::string> > >, boost::regex_traits<char, boost::cpp_regex_traits<char> > >::construct_init(boost::basic_regex<char, boost::regex_traits<char, boost::cpp_regex_traits<char> > > const&, boost::regex_constants::_match_flags)'
collect2: error: ld returned 1 exit status
Makefile:684: recipe for target 'timbl' failed
make[2]: *** [timbl] Error 1
make[2]: Leaving directory '/mnt/b5320167-5dbd-4498-bf34-173ac5338c8d/Tools/timbl-6.4.10/src'
Makefile:465: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/mnt/b5320167-5dbd-4498-bf34-173ac5338c8d/Tools/timbl-6.4.10'
Makefile:374: recipe for target 'all' failed
make: *** [all] Error 2

I'm attaching the full report as "log".

I've used the latest ticcutils-0.17 release, and I'm trying to install frog-0.13.9. Timbl version that is failing is timbl-6.4.10.

Thanks for any suggestions about what might be causing this problem!

Simon Suster

@SimonSuster : ADDED as a GitHub issue.

check works fine, but then says no such file found. Can't find dimin file.

Getting this error, any suggestions please:

 flipswitch@devcenter  /tmp/timbl   master ?  make check
Making check in src
make[1]: Entering directory '/tmp/timbl/src'
make  simpletest
make[2]: Entering directory '/tmp/timbl/src'
g++ -DHAVE_CONFIG_H -I. -I..  -I../include  -std=c++11  -g -O2 -fopenmp -I/usr/include/libxml2 -fopenmp -I/usr/local/include -MT simpletest.o -MD -MP -MF .deps/simpletest.Tpo -c -o simpletest.o simpletest.cxx
mv -f .deps/simpletest.Tpo .deps/simpletest.Po
/bin/bash ../libtool  --tag=CXX   --mode=link g++ -std=c++11  -g -O2 -fopenmp -I/usr/include/libxml2 -fopenmp -I/usr/local/include   -o simpletest simpletest.o libtimbl.la -lxml2 -L/usr/local/lib -lticcutils
libtool: link: g++ -std=c++11 -g -O2 -fopenmp -I/usr/include/libxml2 -fopenmp -I/usr/local/include -o .libs/simpletest simpletest.o  ./.libs/libtimbl.so -lxml2 -L/usr/local/lib /usr/local/lib/libticcutils.so -pthread -fopenmp
make[2]: Leaving directory '/tmp/timbl/src'
make  check-TESTS
make[2]: Entering directory '/tmp/timbl/src'
make[3]: Entering directory '/tmp/timbl/src'
PASS: simpletest
============================================================================
Testsuite summary for timbl 6.4.9
============================================================================
# TOTAL: 1
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[3]: Leaving directory '/tmp/timbl/src'
make[2]: Leaving directory '/tmp/timbl/src'
make[1]: Leaving directory '/tmp/timbl/src'
Making check in include
make[1]: Entering directory '/tmp/timbl/include'
Making check in timbl
make[2]: Entering directory '/tmp/timbl/include/timbl'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/tmp/timbl/include/timbl'
make[2]: Entering directory '/tmp/timbl/include'
make[2]: Nothing to be done for 'check-am'.
make[2]: Leaving directory '/tmp/timbl/include'
make[1]: Leaving directory '/tmp/timbl/include'
Making check in demos
make[1]: Entering directory '/tmp/timbl/demos'
make[1]: Nothing to be done for 'check'.
make[1]: Leaving directory '/tmp/timbl/demos'
Making check in docs
make[1]: Entering directory '/tmp/timbl/docs'
make[1]: Nothing to be done for 'check'.
make[1]: Leaving directory '/tmp/timbl/docs'
Making check in m4
make[1]: Entering directory '/tmp/timbl/m4'
make[1]: Nothing to be done for 'check'.
make[1]: Leaving directory '/tmp/timbl/m4'
make[1]: Entering directory '/tmp/timbl'
make[1]: Leaving directory '/tmp/timbl'
 flipswitch@devcenter  /tmp/timbl   master ?  timbl -v
timbl: error while loading shared libraries: libtimbl.so.4: cannot open shared object file: No such file or directory

Fixed the error by changing the so dynamic linking conf file:

 flipswitch@devcenter  /tmp/timbl   master ?  cat /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf
 flipswitch@devcenter  /tmp/timbl   master ?  vi /etc/ld.so.conf
 flipswitch@devcenter  /tmp/timbl   master ?  sudo vi /etc/ld.so.conf
flipswitch@devcenter  /tmp/timbl   master ?  cat /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf
/usr/local/lib
 flipswitch@devcenter  /tmp/timbl   master ?  timbl -v               
TiMBL 6.4.9 (c) CLST/ILK/CLIPS 1998 - 2017.
Tilburg Memory Based Learner
Centre for Language and Speech Technology, Radboud University
Induction of Linguistic Knowledge Research Group, Tilburg University
CLiPS Computational Linguistics Group, University of Antwerp
Tue Feb 21 21:40:45 2017

option-error: missing value for option ''v'
usage:  timbl -f data-file {-t test-file}
or see: timbl -h
        for all possible options

Now, can't file dimin.train