reneklacan / symspell Goto Github PK
View Code? Open in Web Editor NEWSpelling correction & Fuzzy search based on Symmetric Delete spelling correction algorithm.
License: MIT License
Spelling correction & Fuzzy search based on Symmetric Delete spelling correction algorithm.
License: MIT License
Hello @reneklacan, you did a tremendous work on this !! I need some clarifications on this,
Thanks in Advance..)
I would like a way to make this crate use triple_accel's Levenshtein distance instead of strsim's, since triple_accel's has an option to use simd, and set a maximum distance, rather than doing a full computation and discarding the result. This should make it quite a bit faster again.
Hello @reneklacan, I'm Implemented a Python bindings for this package.For that, I made a separate folder. Could I make a PR for that?
Hey, thanks for creating this port, this is super useful for me!
Once thing that I would appreciate is an ability to silence the output. I'd like to use this tool in another library but I will be calling it many, many times. Getting all of these progress
messages for every query would be a big pain.
If you used the log
crate instead of eprintln
it would be easy to silence those messages (if you want to keep them at all).
What is the status of this project? Are you interested in PRs?
Code to reproduce:
/src/main.rs
extern crate symspell;
use symspell::{AsciiStringStrategy, SymSpell};
fn main() {
let mut symspell: SymSpell<AsciiStringStrategy> = SymSpell::default();
symspell.load_dictionary("data/frequency_dictionary_en_82_765.txt", 0, 1, " ");
symspell.load_bigram_dictionary(
"./data/frequency_bigramdictionary_en_243_342.txt",
0,
2,
" "
);
let sentence = "heymasdaasdnasdhfdgotuyjkwareyou";
let compound_suggestions = symspell.lookup_compound(sentence, 2);
println!("this causes a panic from multiplication overflow:");
println!("{:?}", compound_suggestions);
}
Cargo.toml
[package]
name = "symspell_overflow_test"
version = "0.0.1"
authors = ["Your name <[email protected]>"]
[dependencies]
symspell = "0.4.0"
cargo build && cargo run
ends with:
Running `target/debug/symspell_overflow_test`
thread 'main' panicked at 'attempt to multiply with overflow', /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libcore/num/mod.rs:1938:28
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
So then I ran RUST_BACKTRACE=full cargo run
and got
$ RUST_BACKTRACE=full cargo run
Compiling symspell_overflow_test v0.0.1 (/Users/sh/q/scratch/rust_symspell)
Finished dev [unoptimized + debuginfo] target(s) in 3.85s
Running `target/debug/symspell_overflow_test`
thread 'main' panicked at 'attempt to multiply with overflow', /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libcore/num/mod.rs:1938:28
stack backtrace:
0: 0x10fdcacff - backtrace::backtrace::libunwind::trace::he11f30ca242db87a
at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
1: 0x10fdcacff - backtrace::backtrace::trace_unsynchronized::h94ee7e02d675d126
at /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
2: 0x10fdcacff - std::sys_common::backtrace::_print_fmt::h37608594d2ca99cd
at src/libstd/sys_common/backtrace.rs:78
3: 0x10fdcacff - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h248f4d24421ce424
at src/libstd/sys_common/backtrace.rs:59
4: 0x10fde2d0e - core::fmt::write::h71c869ef05998146
at src/libcore/fmt/mod.rs:1069
5: 0x10fdc9407 - std::io::Write::write_fmt::h6c18aa5d536af338
at src/libstd/io/mod.rs:1427
6: 0x10fdccaca - std::sys_common::backtrace::_print::h9c7302dcb6bc9d80
at src/libstd/sys_common/backtrace.rs:62
7: 0x10fdccaca - std::sys_common::backtrace::print::hef121cec42b01afc
at src/libstd/sys_common/backtrace.rs:49
8: 0x10fdccaca - std::panicking::default_hook::{{closure}}::h718f6688c2c8071f
at src/libstd/panicking.rs:198
9: 0x10fdcc80c - std::panicking::default_hook::h559165f971f7c4f9
at src/libstd/panicking.rs:218
10: 0x10fdcd038 - std::panicking::rust_panic_with_hook::h58f6a2672cadd86f
at src/libstd/panicking.rs:511
11: 0x10fdccc02 - rust_begin_unwind
at src/libstd/panicking.rs:419
12: 0x10fde834f - core::panicking::panic_fmt::h599bb93f00742409
at src/libcore/panicking.rs:111
13: 0x10fde82a7 - core::panicking::panic::hbb1c51e4345889f0
at src/libcore/panicking.rs:54
14: 0x10fd8e960 - core::num::<impl i64>::pow::h49983a2eaaa29165
at /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libcore/num/mod.rs:1938
15: 0x10fda7564 - symspell::symspell::SymSpell<T>::lookup_compound::h77e9a357a7b349df
at /Users/sh/.cargo/registry/src/github.com-1ecc6299db9ec823/symspell-0.4.0/src/symspell.rs:615
16: 0x10fd9848c - symspell_overflow_test::main::h4a6f249518fcfc2f
at src/main.rs:18
17: 0x10fd8c12e - std::rt::lang_start::{{closure}}::h1d323437a9a0069f
at /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libstd/rt.rs:67
18: 0x10fdcd414 - std::rt::lang_start_internal::{{closure}}::h9e16343c6c2438e4
at src/libstd/rt.rs:52
19: 0x10fdcd414 - std::panicking::try::do_call::h3e909382cd5fe48d
at src/libstd/panicking.rs:331
20: 0x10fdcd414 - std::panicking::try::ha497cd00ed772379
at src/libstd/panicking.rs:274
21: 0x10fdcd414 - std::panic::catch_unwind::h798c76fd47e59915
at src/libstd/panic.rs:394
22: 0x10fdcd414 - std::rt::lang_start_internal::h70084482207aef9f
at src/libstd/rt.rs:51
23: 0x10fd8c111 - std::rt::lang_start::h1483ded7753c42a1
at /rustc/f509b26a7730d721ef87423a72b3fdf8724b4afa/src/libstd/rt.rs:67
24: 0x10fd985c2 - symspell_overflow_test::main::h4a6f249518fcfc2f
So the error is on this line,
let si_count: f64 = 10f64
/ ((10i64).pow(self.string_strategy.len(&term_list1[i]) as u32)) as f64;
Hi @reneklacan, really appreciate this Rust repository.
Just one question, is WordSegmentation not implemented in this version?
Thanks!
load_dictionary took 0.035 ms
[]
lookup took 0.022 ms
[Suggestion { term: "whereis th elove hehad dated forimuch of thepast who couqdn'tread in sixtgrade and ins pired him", distance: 1, count: 0 }]
lookup_compound took 14.543 ms
[Suggestion { term: "the bigjest playrs in te strogsommer film slatew ith plety of funn", distance: 0, count: 0 }]
lookup_compound took 5.505 ms
Is it possible for you to make iOS port from Rust?
I try to create it but as I do not have knowledge of Rust I am not able to complete it.
If we can create this iOS port, we can help many needed iOS developers to use this great symspell library.
I can share you my demo project(iOS-Rust for SymSpell) with you.
https://drive.google.com/file/d/19vMeStYUWOZpEYtdybZMi_VZY9-EJmvG/view?usp=sharing
Can you lease make required changes so we can use directly to iOS project?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.