Comments (2)
PSeitz
commented on June 8, 2024
Having a different tokenizer for the fast-field is a good idea, but I would make it configurable and not fix it to raw. There also may be scenarios, where dictionary duplication is wasteful.
from tantivy.
fulmicoton
commented on June 8, 2024
@PSeitz I think we should not care about dictionary duplication here. Configurable tokenizer is ok for me. It really don't believe in actual tokenization (as in splitting the string into tokens) but it might be useful for normalization maybe?
from tantivy.
Related Issues (20)
- Panic occured while committing writer with sort by text field. HOT 6
- Index encryption via Encrypted Directory [reopen]
- Requesting tantivy release to support zstd dependency HOT 9
- Panic on document of death
- MemorySanitizer: use-of-uninitialized-value (probably false positive)
- Top Hits Agg Performance: Delay fetching `get_document_field_data` in top_hits
- top_hits aggregation parameter validation HOT 1
- add `top_metrics` aggregation
- Planned removal of index sorting in 0.23.0 HOT 4
- Regular Expression Queries HOT 1
- Occur - Phrase query with slop HOT 1
- Negative Term HOT 2
- FuzzyTermQuery, PhrasePrefixQuery, PhraseQuery --> Snippets / actual matches
- tantivy vs Lucene benchmark confusion HOT 5
- Questions about the details of how Tantivy manages index files. HOT 2
- Replace PreTokenizedString
- support multiple modes of execution for fastfield range query
- Addition of a migration guide from Lucene to Tantivy HOT 2
- Store DateTime as nanos in docstore HOT 2
- QueryParser raises SyntaxError when combining field prefix with nested searchstring
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tantivy.