reneklacan / symspell Goto Github PK

View Code? Open in Web Editor NEW

130.0 8.0 29.0 2.6 MB

Spelling correction & Fuzzy search based on Symmetric Delete spelling correction algorithm.

License: MIT License

Rust 100.00%

symspell rust spelling-correction spellcheck

symspell's People

Contributors

Stargazers

Watchers

symspell's Issues

SymSpell.WordSegmentation

Hi @reneklacan, really appreciate this Rust repository.

Just one question, is WordSegmentation not implemented in this version?

Thanks!

Use `triple_accel` instead of `strsim` to make it faster

I would like a way to make this crate use triple_accel's Levenshtein distance instead of strsim's, since triple_accel's has an option to use simd, and set a maximum distance, rather than doing a full computation and discarding the result. This should make it quite a bit faster again.

levenshtein_simd_k_with_opts

Python Bindings for Symspell-Rust Package?

Hello @reneklacan, I'm Implemented a Python bindings for this package.For that, I made a separate folder. Could I make a PR for that?

should demo.rs dump corrected result?

load_dictionary took 0.035 ms
[]
lookup took 0.022 ms
[Suggestion { term: "whereis th elove hehad dated forimuch of thepast who couqdn'tread in sixtgrade and ins pired him", distance: 1, count: 0 }]
lookup_compound took 14.543 ms
[Suggestion { term: "the bigjest playrs in te strogsommer film slatew ith plety of funn", distance: 0, count: 0 }]
lookup_compound took 5.505 ms

Python Bindings for Rust Crate?

Hello @reneklacan, you did a tremendous work on this !! I need some clarifications on this,

Is there any Python Bindings available for Rust Crate? And what is it?
Do you have any plan for providing such python bindings?

Thanks in Advance..)

Suggestion: use `log`

Hey, thanks for creating this port, this is super useful for me!

Once thing that I would appreciate is an ability to silence the output. I'd like to use this tool in another library but I will be calling it many, many times. Getting all of these progress messages for every query would be a big pain.

If you used the log crate instead of eprintln it would be easy to silence those messages (if you want to keep them at all).

What is the status of this project? Are you interested in PRs?

iOS port for SymSpell

Is it possible for you to make iOS port from Rust?

I try to create it but as I do not have knowledge of Rust I am not able to complete it.

If we can create this iOS port, we can help many needed iOS developers to use this great symspell library.

I can share you my demo project(iOS-Rust for SymSpell) with you.

https://drive.google.com/file/d/19vMeStYUWOZpEYtdybZMi_VZY9-EJmvG/view?usp=sharing

Can you lease make required changes so we can use directly to iOS project?

Multiplication overflow in lookup_compound for strings with more than 18 characters without spaces

Code to reproduce:

/src/main.rs

extern crate symspell;

use symspell::{AsciiStringStrategy, SymSpell};

fn main() {
    let mut symspell: SymSpell<AsciiStringStrategy> = SymSpell::default();

    symspell.load_dictionary("data/frequency_dictionary_en_82_765.txt", 0, 1, " ");
    symspell.load_bigram_dictionary(
      "./data/frequency_bigramdictionary_en_243_342.txt",
      0,
      2,
      " "
    );


    let sentence = "heymasdaasdnasdhfdgotuyjkwareyou";
    let compound_suggestions = symspell.lookup_compound(sentence, 2);
    println!("this causes a panic from multiplication overflow:");
    println!("{:?}", compound_suggestions);
}

Cargo.toml

[package]
name = "symspell_overflow_test"
version = "0.0.1"
authors = ["Your name <[email protected]>"]

[dependencies]
symspell = "0.4.0"

cargo build && cargo run

ends with:

     Running `target/debug/symspell_overflow_test`
thread 'main' panicked at 'attempt to multiply with overflow', /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libcore/num/mod.rs:1938:28
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

So then I ran RUST_BACKTRACE=full cargo run and got

$ RUST_BACKTRACE=full cargo run
   Compiling symspell_overflow_test v0.0.1 (/Users/sh/q/scratch/rust_symspell)
    Finished dev [unoptimized + debuginfo] target(s) in 3.85s
     Running `target/debug/symspell_overflow_test`
thread 'main' panicked at 'attempt to multiply with overflow', /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libcore/num/mod.rs:1938:28
stack backtrace:
   0:        0x10fdcacff - backtrace::backtrace::libunwind::trace::he11f30ca242db87a
                               at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
   1:        0x10fdcacff - backtrace::backtrace::trace_unsynchronized::h94ee7e02d675d126
                               at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
   2:        0x10fdcacff - std::sys_common::backtrace::_print_fmt::h37608594d2ca99cd
                               at src/libstd/sys_common/backtrace.rs:78
   3:        0x10fdcacff - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h248f4d24421ce424
                               at src/libstd/sys_common/backtrace.rs:59
   4:        0x10fde2d0e - core::fmt::write::h71c869ef05998146
                               at src/libcore/fmt/mod.rs:1069
   5:        0x10fdc9407 - std::io::Write::write_fmt::h6c18aa5d536af338
                               at src/libstd/io/mod.rs:1427
   6:        0x10fdccaca - std::sys_common::backtrace::_print::h9c7302dcb6bc9d80
                               at src/libstd/sys_common/backtrace.rs:62
   7:        0x10fdccaca - std::sys_common::backtrace::print::hef121cec42b01afc
                               at src/libstd/sys_common/backtrace.rs:49
   8:        0x10fdccaca - std::panicking::default_hook::{{closure}}::h718f6688c2c8071f
                               at src/libstd/panicking.rs:198
   9:        0x10fdcc80c - std::panicking::default_hook::h559165f971f7c4f9
                               at src/libstd/panicking.rs:218
  10:        0x10fdcd038 - std::panicking::rust_panic_with_hook::h58f6a2672cadd86f
                               at src/libstd/panicking.rs:511
  11:        0x10fdccc02 - rust_begin_unwind
                               at src/libstd/panicking.rs:419
  12:        0x10fde834f - core::panicking::panic_fmt::h599bb93f00742409
                               at src/libcore/panicking.rs:111
  13:        0x10fde82a7 - core::panicking::panic::hbb1c51e4345889f0
                               at src/libcore/panicking.rs:54
  14:        0x10fd8e960 - core::num::<impl i64>::pow::h49983a2eaaa29165
                               at /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libcore/num/mod.rs:1938
  15:        0x10fda7564 - symspell::symspell::SymSpell<T>::lookup_compound::h77e9a357a7b349df
                               at /Users/sh/.cargo/registry/src/github.com-1ecc6299db9ec823/symspell-0.4.0/src/symspell.rs:615
  16:        0x10fd9848c - symspell_overflow_test::main::h4a6f249518fcfc2f
                               at src/main.rs:18
  17:        0x10fd8c12e - std::rt::lang_start::{{closure}}::h1d323437a9a0069f
                               at /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libstd/rt.rs:67
  18:        0x10fdcd414 - std::rt::lang_start_internal::{{closure}}::h9e16343c6c2438e4
                               at src/libstd/rt.rs:52
  19:        0x10fdcd414 - std::panicking::try::do_call::h3e909382cd5fe48d
                               at src/libstd/panicking.rs:331
  20:        0x10fdcd414 - std::panicking::try::ha497cd00ed772379
                               at src/libstd/panicking.rs:274
  21:        0x10fdcd414 - std::panic::catch_unwind::h798c76fd47e59915
                               at src/libstd/panic.rs:394
  22:        0x10fdcd414 - std::rt::lang_start_internal::h70084482207aef9f
                               at src/libstd/rt.rs:51
  23:        0x10fd8c111 - std::rt::lang_start::h1483ded7753c42a1
                               at /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libstd/rt.rs:67
  24:        0x10fd985c2 - symspell_overflow_test::main::h4a6f249518fcfc2f

So the error is on this line,

let si_count: f64 = 10f64
    / ((10i64).pow(self.string_strategy.len(&term_list1[i]) as u32)) as f64;

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.