The way Node stores string fragments alongside internal fields is problematic:

new Trie(['child']).optimize()
Uncaught TypeError: this.child is not a function
    at Node.isTerminalString (node.js:51)
    at Node.isTerminal (node.js:56)
    at Trie.collapseChains (trie.js:207)
    at Trie.optimize (trie.js:126)
    at <anonymous>:1:26

"child" is a property holding a Node reference, not the child method from the prototype.

This could possibly be worked around by usage of Symbols to insure internal fields/method never conflict with string fragment properties.

Words with underscores are reported as not present in a word collection with isWord

const trie = new Trie(['yes', 'yes_no']);
trie.isWord('yes'); // => true (which is correct)
trie.isWord('no'); // => false (which is correct)
trie.isWord('yes_no'); // => false (which is incorrect)

I guess this is because Node has special properties like _c that use _ and hence any prop starting with _ is excluded in node.props() https://github.com/mckoss/dawg/blob/master/src/node.ts#L88.

I guess a safer option would be to store props in their own dictionary like node.props. Otherwise the only option I can think of is to replace _ prior to creating the Trie and before isWord but that seems less ideal.

I constructed a Trie from a Scrabble dictionary which is then packed and used to construct a PTrie.
The PTrie itself appears to be populated, though methods such as isWord and completions are not returning expected results.

Example:

import { createReadStream, createWriteStream } from 'fs';
import { resolve } from 'path';
import { createInterface, Interface } from 'readline';

var Trie = require('dawg-lookup').Trie;
var PTrie = require('dawg-lookup/lib/ptrie').PTrie;

var trie = new Trie();
var scrabbleDictionary = createInterface({
   input: createReadStream(resolve(__dirname, '../lib', 'Collins Scrabble Words (2019).txt'), 'utf8')
});
var output = createWriteStream(resolve(__dirname, '../lib', 'dictionary.txt'));

scrabbleDictionary.on('line', (line: string) => {
        trie.insert(line);
    }).on('close', () => {
        console.log(trie.isWord('FUN'));

        let packed = trie.pack();
        let ptrie = new PTrie(packed);

        console.log(ptrie.isWord('FUN'));
        console.log(ptrie.completions('F'));

        output.write(packed);
        output.close();
    });

Result:

> true
> false
> []

Any ideas why this is occurring?

Wondering how you arrived at this compression format? Is it based on a research paper or anything, or how much does it compress the basic JSON structure of the Trie class?

How about adding support for arbitrary other scripts, like Chinese text (70,000+ possible characters used by words), or Amharic (~200 or so characters), or other scripts with fewer characters than that. Would it still work out of the box or would the PTrie compression format/algorithm need to be modified?

When you say "Large dictionaries can be further compressed", does that feature come included in the library, or what do you need to do to make that happen?

Also a stretch question, wondering if you could explain more the compression format, for a complex/larger example. I still am not grasping it fully after reading the readme, I think I will have to play around with some data to test it out and learn from experience perhaps, not sure.

Thanks for your time in advance.

P.S. The PTrie algorithm seems so clever, that optimization seems necessary way further down the road from where I'm at in my journey, thanks so much for sharing!

You're probably familiar with agglutinative languages, like Turkish, which can have "sentence words", basically an infinite number of combination of base + n suffixes. Navajo is another example language, or Finnish, Inuktitut, etc.. Turkish words can be at least up to 70 characters, but you never probably want to serialize this word into a trie/dictionary because there would be way too many trie nodes I would imagine.

Sorry to bother you, but you seem like you are a master at this so I thought I'd ask you directly :). How would you architect such a DAWG-like system for Turkish, assuming let's say 10,000 base words, and 1000 suffixes, which can be concatenated infinitely.

Seems like you would not have to serialize/realize the actual TrieNodes, but make it virtualized somehow, and I'm having a tough time imagining how that might work/look. Maybe you have some thoughts or have done this sort of thing before. But it's like, all 11000 fragments (words + suffixes) would be in the central trie. The words would then have as leaves a marker that it was a word itself, plus link to the suffix nodes, so forming a loop. The suffixes would link to suffixes as well. How would that look? Is that something possible even, given the added thing of the "loop"? If it's possible, it doesn't seem straightforward to serialize anymore because of the loops, is it still possible to pack and unpack?

Thanks in advance for the help.

Best,
Lance