mihaivalentin / lunr-languages Goto Github PK

View Code? Open in Web Editor NEW

419.0 18.0 161.0 1.08 MB

A collection of languages stemmers and stopwords for Lunr Javascript library

License: Other

JavaScript 100.00%

lunr-languages stemmer stopwords lunr localization language-stemmer

lunr-languages's People

Stargazers

Watchers

Forkers

ming300 ajinteractive chadliu23 olivernn arve0 indigostudio onevcat buminta leonid-shevtsov yoshidax pjkui htomaka anoochit hunjaejung kolosochek zbingwen weixsong craftzdog polo2ro tommy2z vahidnaderi gitter-badger jdrew1303 fmy hakanerp alexxnica kryndex drzraf keerati danjarvis tanged digitalheir datagenno johndobrien mikalv romgry dekdekbaloo summyd dzeso ilek2428 ahmadibrahiim cuulee ryabchikus leora-betesh not-dalia acedward agentsir krakky softwerkab hgldj1966 cqhaibin shadowcz007 aleksey-golubtsov duck-lab titaniasoftware stownsend2121 mochapanna rikuson martinyounghoonkim sabrack repairearth hotosm billmoyers magicspace martignoni davty fierthraix saqib-ahmed izut shoyuvanilla mikr13 tombarr conoremclaughlin mahdimajidzadeh alifianadexe shumih knubie mqtik dadiorchen duanzhang512 fguby niqingyang jfermat motoitanigaki samandaneshvar darkyzhou nextprops wa4be chokchai9900 aimeiyijia bendawes inquisitorgray kebal2 mochi-cards haghighi thirukkural2022 interweb-lt fauno vitrolom rukeni

lunr-languages's Issues

trimmer

When doing

idx.use(lunr.de)

lunr.trimmer is removed from the pipeline, making words including punctation and the like to enter the index. E.g., both "word." and "word" will enter the index.

Adding lunr.trimmer to the pipeline manually is not really a good solution, as lunr.trimmer uses \W to match non word characters (regexp unicode only supported as of ES6).

A solution could be to normalize characters like æøå -> aoa, like done here: https://github.com/cvan/lunr-unicode-normalizer/blob/master/lunr.unicodeNormalizer.js

Thoughts?

lunr.de fails with umlaute in wildcard search

Searching for words with umlaute fails if wildcard search is used:

const lunr = require('lunr')
require('lunr-languages/lunr.stemmer.support')(lunr)
require('lunr-languages/lunr.de')(lunr)

const idx = lunr(function () {
  this.use(lunr.de)
  this.field('text') 
  this.add({
    id: 1,
    text: 'das ist günstig'
  })
})

console.log(idx.search('günstig').length) // result: 1
console.log(idx.search('günsti*').length) // result: 0

I tried a workaround described in a comment of #66, but this didn't work.
I would be very happy if the bug could be fixed in lunr.de

Support for position metadata in Japanese

Starting with Lunr 2.0.0 you can opt in to use additional search result metadata. If you take a look at the latest demo app you can see how to leverage the new match position metadata.

I am using this feature for English search, however the position metadata is returned as null when searching against the Japanese index. I haven't had a chance to take a look into the details to try and fix it myself. I am submitting this issue here for visibility. I'll update it when I have more information.

Thank you!

French 'Diabète' is not in the index

I have "Diabète" in both title, and body, why it does not appear in the search index?

Update nodejs version in CI

Version 10.xx is EOL

Bower

Please add this repository in the Bower. This will help save time during development.

Another language

Please provide some help on creating stemmer for another language.
I want to create one for Persian (Farsi).
I don't know where to start.

Can "nodejieba" be as "devDependency" or "peerDependency" in lunr-languages?

Main reason for this - due to this dependency package lunr-languages is not working on node.js 16:

gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CXX(target) Release/obj.target/nodejieba/lib/index.o
In file included from ../lib/index.cpp:1:
In file included from ../lib/nodejieba.h:4:
In file included from ../lib/utils.h:4:
In file included from /Users/leo/Library/Caches/node-gyp/16.3.0/include/node/node.h:63:
In file included from /Users/leo/Library/Caches/node-gyp/16.3.0/include/node/v8.h:30:
/Users/leo/Library/Caches/node-gyp/16.3.0/include/node/v8-internal.h:452:38: error: no template named 'remove_cv_t' in namespace 'std'; did you mean 'remove_cv'?
            !std::is_same<Data, std::remove_cv_t<T>>::value>::Perform(data);
                                ~~~~~^~~~~~~~~~~
                                     remove_cv
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/v1/type_traits:776:50: note: 'remove_cv' declared here
template <class _Tp> struct _LIBCPP_TEMPLATE_VIS remove_cv
                                                 ^
1 error generated.
make: *** [Release/obj.target/nodejieba/lib/index.o] Error 1

Minified version of the Thai language is missing

https://github.com/MihaiValentin/lunr-languages/tree/master/min

From above link, there is no Thai. It seems lunr.th.min.js is missing.

Best regards,

Add latest build to npm

I see that the version of lunr-languages on npm differs with the version in this repository (especially the multilanguage support), can you please push the new version to npm?

"multiLanguage" method doesn't work with Japanese.

Hi, I think "multiLanguage" method doesn't work with Japanese.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>Lunr multi-language demo</title>
    <script src="./src/lunr/lunr.js"></script>
    <script src="./src/lunr-languages/lunr.stemmer.support.js"></script>
    <script src="./src/lunr-languages/tinyseg.js"></script>
    <script src="./src/lunr-languages/lunr.ja.js"></script>
    <script src="./src/lunr-languages/lunr.multi.js"></script>
  </head>
  <body>
    <p>Open developer tools and observe the results in the Console tab. View source for code.</p>
    <script>
      /* init lunr */
      var idxEn = lunr(function () {
        this.field('body')
        this.add({"body": "この文章は日本語で書かれています。", "id": 1})
        this.add({"body": "This text is written in the English language.", "id": 2})
      });
      var idxJp = lunr(function () {
        this.use(lunr.ja);
        this.field('body')
        this.add({"body": "この文章は日本語で書かれています。", "id": 1})
        this.add({"body": "This text is written in the English language.", "id": 2})
      });
      var idxMulti = lunr(function () {
        this.use(lunr.multiLanguage('en', 'ja'));
        this.field('body')
        this.add({"body": "この文章は日本語で書かれています。", "id": 1})
        this.add({"body": "This text is written in the English language.", "id": 2})
      });
      console.log('Search for `日本語` (English pipeline): ', idxEn.search('日本語'));
      console.log('Search for `languages` (English pipeline): ', idxEn.search('languages'));
      console.log('Search for `日本語` (Japanese pipeline): ', idxJp.search('日本語'));
      console.log('Search for `languages` (Japanese pipeline): ', idxJp.search('languages'));
      console.log('Search for `日本語` (Jp + En pipeline): ', idxMulti.search('日本語'));
      console.log('Search for `languages` (Jp + En pipeline): ', idxMulti.search('languages'));
    </script>
  </body>
</html>

And the console showed like this.

1 Search for 日本語 (English pipeline): Array []
2 Search for languages (English pipeline): Array [ {…} ]
3 Search for 日本語 (Japanese pipeline): Array [ {…} ]
4 Search for languages (Japanese pipeline): Array []
5 Search for 日本語 (Jp + En pipeline): Array []
6 Search for languages (Jp + En pipeline): Array [ {…} ]

I guess "5" should return Array [ {…} ].

I've tried demo and it worked.
And the console showed like this.

1 Search for Русских (English pipeline): Array []
2 Search for languages (English pipeline): Array [ {…} ]
3 Search for Русских (Russian pipeline): Array [ {…} ]
4 Search for languages (Russian pipeline): Array []
5 Search for Русских (Ru + En pipeline): Array [ {…} ]
6 Search for languages (Ru + En pipeline): Array [ {…} ]

Thank you.

Load an index with a custom pipeline

Hey,

I'm having a Error: Cannot load un-registered function: trimmer-fr when trying to load an index built with french stemmer/stopWords.

I have built the index following the doc:

var lunr = require('./lunr.min.js'), fs = require('fs');
require('./lunr.stemmer.support.js')(lunr);
require('./lunr.fr.js')(lunr);

var idx = lunr(function() {
  this.use(lunr.fr);

// ...
});

Then I load the index like this:
lunr.Index.load(idx);

I'm doing this because I have the index file on a static host environment and I am fetching it in my mobile app. This way I don't have to bundle my app with it.
For now, to make it works, I'm replacing the pipeline with the default one when I fetch it.
idx.pipeline = idx.pipeline.map(pipe => pipe.replace('-fr', ''))

Am I breaking something if I do this?
Is there any other workarounds/fix?

Thanks!

Activate plugin in Reactjs

How can I add this support in Reactjs where ES2015 and ES2016 are used.
Where modules are not required but imported.

tried these but didn't work

import lunr from 'lunr';
require('../node_modules/lung-languages/lunr.stemmer.support.js')(lunr)

import lunr from 'lunr';
import {} from 'lunr-languages';

var idx = lunr(function(){
   this.use(require('../node_modules/lung-languages/lunr.stemmer.support.js'))
});

When will the lunr languages version be compatible with lunrjs v.2.3.5?

Hi, when will the lunr languages version be compatible with lunrjs v.2.3.5?

Do you have plans to update it soon?

Thanks.

Fix Dutch language code

I believe the language code for Dutch should be nl and not du according to ISO 639-1, which is the standard linked to in the Language Support guide of lunr.

Use stopwords from multiple languages

Someone asked me by email this question, so I'll add it here perhaps this may help others as well:

Is it possible to use multiple languages for stopwords with your lunr extension? In my case german, french and english in nodejs, like this:

global.lunr = require('./lib/lunr.js');
require('./lunr.stemmer.support.js');
require('./lunr.de.js');
require('./lunr.fr.js');

var idx = lunr(function () {
    this.use(lunr.de);
    this.use(lunr.fr);
    this.field('title', { boost: 10 })
    this.field('body')
});

Not able to search for just numbers in lunr.de

Probem

In my German and English test documents I have content with the term Port 1234, but searching for 1234 does not work.

Has someone seen the same or a similar problem? Any ideas?

More tests

Searching for Port 1234 works fine.
Searching for 1234 in an English document works fine, using the base lunr.js version 2.3.8
Using this.use(lunr.de) makes it possible to find German umlauts but no numbers anymore.

Test Code

// The required JS files are correctly inserted in the sites head

var idx = lunr(function () {
  this.use(lunr.de)
  this.ref('id')
  this.field('text')

  this.add({
    id: 1,
    text: "Port 1234 is a good port for testing a problem"
  })
})

console.log(idx.search('1234'));
console.log(idx.search('Port 1234'));

Result

Can anybody release lunr-language file for Persian or Arabic languages please?

I don't have enough knowledge to write the lunr-language for Persian or Arabic languages.
Persian language reference is: https://fa.wikipedia.org/wiki/%D8%A7%D9%84%D9%81%D8%A8%D8%A7%DB%8C_%D9%81%D8%A7%D8%B1%D8%B3%DB%8C
and if you are required more information about Persian (Farsi) language, I'd like to provide a lot.
Thank you!

How do I go about adding chinese and korean support?

Please advise. The instruction isn't so clear on how to add additional language support.

Problem in spanish: doesn't work if word isn't using accent mark.

Let's say I have an index created. the spanish word "Respiración" is stemmed as:
"respir"

Thats correct.

Now, I make a search, but the user doesn't use the accent mark, and he types: "respiracion" (without acent on last "o"). So lunr won't stem that word and it will let it as "respiracion", so no matches will be found.

I know that a basis around stemming is that the word is correctly spelled, BUT as nearly no user type accents correctly when searching for a string, this is really making lunr useless for many words.

Hindi Language Support required

Opening this ticket as I'm looking for support of Hindi Language.

Add Indonesia as supported language

Opening this ticket as I'm looking to add Bahasa Indonesia as a supported language. At https://github.com/hotosm we're working on various documentation sites that need support across a number of languages, including Indonesian.

If anyone has already worked on this, please chime in. Otherwise, we will be looking to add the support and create a PR when ready.

Esperanto Support

Hello, I would like to add support for the Esperanto language, as several projects downstream I use depend on lunr-languages. It is a constructed languaged invented in 1887 by Dr. L.L. Zamenhof, and has over 2-million speakers worldwide.

Fortunately due to the extreme regularity of the language (it only has 16 rules), implementing this should be a lot easier than for other languages.

Advice Needed:

I don't normally work with JavaScript, so I was wondering if anyone involved with the project can help me out with a few things:

Does the stop-words function run before the stemmer? It would greatly reduce the burden if stop-words are filtered out before they get to the stemmer. Otherwise, I will basically wind up having to reimplement the stop-words list again in the stemmer, as most of the stop-words are grammatical prepositions and the like that have irregular endings.
Many other languages have very complicated hundred-line stemmer functions, but in Esperanto, once you filter the special grammatical words, every word ends with either: -is, -as, -os, -us, -u, -e, -en, -a, -an -aj, -ajn, -o, -on, -oj, or -ojn. With that said, my stemmer function can be as simple as just returning a string with the end cut off (this always results in a valid word root). I wasn't sure if I needed to use the SnowballFunction or not.

I'm currently working on Esperanto support on my fork if anyone has any advice, or wants to point out any obvious JS flaws I missted.

TypeDefinition for this library?

Will lunr-language allow typescript imports to import "lunr-languages/lunr.stemmer.support", "lunr-languages/lunr.multi" and "lunr-languages/lunr.<locale>"?

The workaround is to require(....)(lunr) to add functions to the prototype of lunr and call them in typescript code.

I am not sure how to write the corresponding typedef.d.ts files nor can they be port to typedef files at all.
Please kindly advise.

React JS

Hi I am using lunr, and would love to use the italian language support. Lunr works great with React. Is it possible to use the italian plugin with react? How would I do that?

Thank you for your help!

Kind regards,

YesSeri

How to do use it in jekyll blog?

I finished jekyll-lunr-js-search https://github.com/slashdotdash/jekyll-lunr-js-search, but didn't understand how to build the lunr-languages in my blog.
my search.md is :

---
layout: default
title: Search
permalink: /search/

---
<div id="search">
  <form action="/search" method="get">
    <input type="text" id="search-query" name="q" placeholder="Search" autocomplete="off">
  </form>
</div>

<script src="/js/jquery-1.9.1.min.js" type="text/javascript" charset="utf-8"></script>
<script src="/js/require.js"></script>
<script src="/js/lunr.stemmer.support.js"></script>
<script src="/js/lunr.jp.js"></script>
<script src="/js/lunr.min.js" type="text/javascript" charset="utf-8"></script>
<script src="/js/mustache.js" type="text/javascript" charset="utf-8"></script>
<script src="/js/date.format.js" type="text/javascript" charset="utf-8"></script>
<script src="/js/URI.min.js" type="text/javascript" charset="utf-8"></script>
<script src="/js/jquery.lunr.search.js" type="text/javascript" charset="utf-8"></script>

<section id="search-results" style="display: none;">
  <p>Search results</p>
  <div class="entries">
  </div>
</section>

<script type="text/javascript">
  $(function() {
    $('#search-query').lunrSearch({
      indexUrl  : '/js/index.json',           // url for the .json file containing search index data// URL of the `search.json` index data for your site
      results   : '#search-results',          // selector for containing search results element// jQuery selector for the search results container
      template  : '#search-results-template', // selector for Mustache.js template// jQuery selector for the Mustache.js template
      titleMsg  : '<h1>Search results<h1>',   // message attached in front of results (can be empty)
      emptyMsg  : '<p>Nothing found.</p>'     // shown message if search returns no results
      entries:  '.entries',                 // jQuery selector for the element to contain the results list, must be a child of the results element above.   
    });
  });
</script>

<script src="/js/search.min.js" type="text/javascript" charset="utf-8"></script>

<script type="text/javascript">
var idx = lunr(function () {
    // use the language (jp)
    this.use(lunr.jp);
    // then, the normal lunr index initialization
    this.field('title', { boost: 10 });
    this.field('body');
});
</script>
<script type="text/javascript">
require(['js/lunr.js', 'js/lunr.stemmer.support.js', 'js/lunr.jp.js'], function(lunr, stemmerSupport, jp) {
    // since the stemmerSupport and de add keys on the lunr object, we'll pass it as reference to them
    // in the end, we will only need lunr.
    stemmerSupport(lunr); // adds lunr.stemmerSupport
    jp(lunr); // adds lunr.de key
    // at this point, lunr can be used
    var idx = lunr(function () {
        // use the language (jp)
        this.use(lunr.jp);
        // then, the normal lunr index initialization
        this.field('title', { boost: 10 })
        this.field('body')
    });
});
</script>
{% raw %}
<script id="search-results-template" type="text/mustache">
  {{#entries}}
    <article>
      <h2>
        {{#date}}<small><time datetime="{{pubdate}}" pubdate>{{displaydate}}</time></small>{{/date}}
        <a href="{{url}}">{{title}}</a>
      </h2>
      {{#is_post}}
      <ul>
{{#tags}} <small> {{.}} </small>{{/tags}}
      </ul>
      {{/is_post}}
    </article>
  {{/entries}}
</script>
{% endraw %}

please help me, thank you.

Add source repository information to metadata

The current npm package (0.0.4) cannot be deployed as a WebJar because it's missing the source repository in the metadata. It would be great if this could be added in the next release.

Kazakh support

I would like to implement the Kazakh language. The only stemmers I've been able to find, though, are simple suffix removers, like this: https://github.com/maximgorbatyuk/Kazakh-Stemmer-Elasticsearch-Plugin/blob/master/src/main/java/org/elasticsearch/plugin/KazakhStemmerTokenFilter.java

I'm going to get my feet wet with Armenian (#60) but hope to tackle this next. As always, if anyone has already done this, I welcome any information.

Need Chinese support

Arabic

Salut,

Last autumn I hacked this Arabic Stemmer https://github.com/ejtaal/jsastem into a fork of your project https://github.com/AJInteractive/lunr-languages for PalestineRemix.com

I would like to revise how I did that, clean it up and contribute back; So, does this https://github.com/AJInteractive/lunr-languages/blob/master/lunr.ar.js look sane to you or I need to provide that functionality in a different way?

Also I need find ways to add support for Bosnian (or Croatian).

Cheers,
Laurian

Ability to use without having global.lunr

Nice module, and definitely a missing functionality in Lunr.

I would like to us this plugin with pouchdb-quick-search, but it's a bit awkward that I need a global window.lunr or process.lunr to be defined in order for it to work. pouchdb-quick-search uses browserify, so there's no need to rely on global window variables.

It'd be nice if, in the Node version at least, this plugin could be used without needing to modify the global namespace. Then I could use browserify when I build for the browser, even if this module doesn't use browserify.

Connection problem between problemlunr.ja.js tinyseg

Hi, Thank you for this awesome project!

I'm using lunr with Japanese text, so I write code like this

var lunr = require('lunr');
require('./language/lunr.stemmer.support.js')(lunr);
require('./language/lunr.jp.js')(lunr);

var idx = lunr(function() {
  this.use('lunr.jp');
  this.addField('title');
  this.addField('body');
  this.setRef('id');
})

But when I execute script, error occur with message
"TypeError: lunr.TinySegmenter is not a function"
I think there are some connection problems between lunr.jp.js and tinyseg.js.
I'm not familliar with JS, so I don't know how to fix this problem.
Can someone helps me?

Accented letter ê should be replaced by e in the french stemmer

Currently, "empêchaient" (verb "empêcher" conjugated in past) will be indexed as "empêch" (instead of "empech").

I'm not familiar with http://snowball.tartarus.org/ nor stemmer algorithms but according to http://snowball.tartarus.org/algorithms/french/stemmer.html this is the expected behavior.
For instance, maître will produce maîtr not maitr. I find it odd, because most of the time French people will not type accented letters when searching (because it's quicker to type and most search engine will replace accented letters anyway).

For reference, here's the Lucene implementation: https://github.com/apache/lucene-solr/blob/master/lucene/analysis/common/src/java/org/apache/lucene/analysis/fr/FrenchLightStemmer.java

Initialize multi-languages for pouchdb-quick-search

I've open the following issue pouchdb-community/pouchdb-quick-search#56 because I'm not exactly sure how to initialize multi languages for this project (pouchdb-quick-search) in pure javascript... I've checked his test but he seems to use requireJS!

Any help would be highly appreciated

Dutch stopwords

Hi, I've been trying out your great Lunr extension.
It seems to work for the most part except that dutch stopwords (in my case "en") do not seem to be ignored.
Do you have any idea what the problem might be?

Thank you

problem with FR

Hello i've tried integrate lunr-fr in my project and i have this message in console

lunr.SortedSet is not a constructor

my lunr version is 2.3.6

Please update npm package

I've seen @keerati added file lunr.th.js to support Thai language since Jun 2017, but this file doesn't include in thevlatest version of your npm package (1.0.0)

jp plugin doesn't support umd

Since /tinyseg.js related to Japanese plugin does not support umd, it can not be used with node.js or require.js.

I fixed this problem, so I will pull the request. 😃

nodejieba (Chinese) is not working with webpack

When using webpack and lurn.zh, it will results in:

TypeError: nodejieba.cut is not a function
    at lunr.Builder.lunr.zh.tokenizer (webpack://test/./node_modules/lunr-languages/lunr.zh.js?:92:17)
    at lunr.Builder.add (webpack://test/./node_modules/lunr/lunr.js?:2479:23)
    ....

It seems that nodejieba is minimized by webpack and "cut" is no longer a valid function name.

I've tried:

 optimization: {
    minimize: false
  }

But it didn't resolve the issue. Still trying to figure out a solution...

require without full path

It would be nice if one could require lunr-languages without specifying the path.

E.g.:

var lunrLang = require('lunr-languages');
var lunrNo = lunrLang.no;

lunr.de demo with unexpected result for umlauts

I tried your demosite "demo-browser-require.html", but I don't understand the results.

tests:
console.log('Search for günstige: ', idx.search('günstige'));// expected resultsize: 1, result: 1
console.log('Search for günstig*: ', idx.search('günstig*'));// expected resultsize: 1, result: 0
console.log('Search for g*nstig*: ', idx.search('g*nstig*'));// expected resultsize: 1, result: 1

source: https://rawgit.com/MihaiValentin/lunr-languages/master/demos/demo-browser-require.html

Did I missunderstood, how to search for words with umlauts, or is it not possible to search with wildcards for words with umlauts?

Japanese minified versions missing

lunr.ja.min.js and lunr.jp.min.js are missing.

How can I add Chinese support

I want to make my metalsmith-lunr support Chinese, How can I do it?

I would like to add korean language feature

I have been looking for how to add search feature on my jekyll blog which is written in Korean. and I come across this issue tells about how to add language support.

I would like to add korean language this project. so could you tell more detail how do i contribute?

lunr.TinySegmenter is not a constructor on lunr.ja file

Hello, I require your help if someone already saw this problem.
I use mkdocs and so to build the search I specify the lang with ja.
I tried with 5 or 6 different language, all work except this one, this error is show all the time:
lunr.TinySegmenter is not a constructor
at lunr.ja.js:86
at lunr.ja.js:36
at lunr.ja.js:38
at loadScripts (worker.js:29)
at XMLHttpRequest.onJSONLoaded (worker.js:53)
Does someone already saw this error and found a way to fix it?

I'm still working on it so if I find the solution I will post it.
Thank you

Use lunr.de with requirejs

Hello,
I can't figure out how to use lunr.de with requirejs. I tried :

CASE 1

paths: {
    lunr        : 'path/to/lunr'
    lunrde      : 'path/to/lunr.de' 
},
shim {
    lunr: {
        exports: 'lunr'
    },
    lunrde: {
        deps: ['lunr'],
        exports: 'lunrde'
    }
}

And in another file

define(['app',
        'lunr',
        'lunrde'
],
function(App, lunr, lunrde) {
    var functionName = function() {
        idx = lunr(function() {
            this.use(lunrde);
        });
    }
}

CASE 2

paths: {
    lunr        : 'path/to/lunr'
    'lunr.de'    : 'path/to/lunr.de' 
},
shim {
    lunr: {
        exports: 'lunr'
    },
    'lunr.de': {
        deps: ['lunr'],
    }
}

And in another file

define(['app',
        'lunr',
        'lunrde'
],
function(App, lunr) {
    var functionName = function() {
        idx = lunr(function() {
            this.use(lunr.de);
        });
    }
}

In both cases (and all mixes between them), I'll keep have the exception 'Lunr is not present. Please include / require Lunr before this script.' and another error from lunr because he couldn't find "lunr.de"... It's work with lunr alone !

Do someone know how to include lunr globally with requirejs or how to use lunr.de with require js ? (For brevity here, I don't include lunr.stemmer.support).

Usage of Lunr js with Turkish language

I am trying to use Lunr library in my nodejs environment. Here is the code block that makes the basic search.

const lunr = require("lunr");
require('lunr-languages/lunr.stemmer.support.js')(lunr);
require('lunr-languages/lunr.tr.js')(lunr);
    
var idx = lunr(function () {
    this.use(lunr.tr);
    this.ref('name');
    this.field('text');
    this.metadataWhitelist = ['position'];
    this.add({
        "name": "./file1.txt",
        "text": "türkçe"
    });
    this.add({
        "name": "./file2.txt",
        "text": "kullanıcı"
    });
});

function searchFor(token) {
    let searchResult = idx.search(`*${token}*`);
    console.log(searchResult.length);
}

searchFor("türkçe")
searchFor("kullanıcı")

The first search hits 1 result as expected. However the second one finds no match. I wonder the reason behind. I have tried using the multi-language as well by adding this.use(lunr.multiLanguage("tr")); and removing this.use(lunr.tr);. Also I tried removing the wildcards * from my search that hits a result but that is not the scenario I need. Is it a problem with the 'tr' support or there is a misunderstanding with my usage?

Armenian support

I would like to implement Armenian support, and I see that there is a Java stemmer in the jssnowball project: https://github.com/mazko/jssnowball/blob/master/snowball_cache/snowball_all/algorithms/armenian/stemmer.java

I'm new to this whole thing, so if anyone has already done this, I would greatly appreciate any tips. Otherwise, I'll get hacking.

lunr-languages/lunr.fr.js fails to find common words like "équipement"

Test case:

import lunr from "lunr"
import stemmerSupport from 'lunr-languages/lunr.stemmer.support.js'
import lunrfr from 'lunr-languages/lunr.fr.js'

stemmerSupport(lunr)
lunrfr(lunr)

const docs = [
    {
        text : "équipement, barrage",
        id: '1'
    },
    {
        text : "rivière",
        id: '2'
    }
]

const index = lunr(function () {
    this.field('text')
    this.ref('id')

    for(const doc of docs){
        this.add(doc)
    }
})

console.log('résultats pour "équipement"', index.search('équipement'))
console.log('résultats pour "barrage"', index.search('barrage'))
console.log('résultats pour "rivière"', index.search('rivière'))

All 3 console.log should return a result, but the first one does not

Hebrew support

Are there any plans to support Hebrew language? Can anyone share their working code?