Giter VIP home page Giter VIP logo

Comments (4)

mschoch avatar mschoch commented on August 16, 2024

Sure, so one of the last things I did for the Bleve project before leaving was to prototype something I called "index sections". The code for this is in this commit: blevesearch/bleve@674c535

The basic idea (as I recall, I have not read the details of this commit recently) was to introduce a new concept to the bleve index. Today, for a field there is just a single type of index, the inverted index. Our idea was that there should be support for other indexes, but because of the design of Bleve, we allow people to mix different data types in fields. So, we can't just say field X is type Y and use the appropriate index, because at runtime someone can hand us some document with a different type for that field.

Anyway, the idea was that all of the existing code, folds into an implementation of a "inverted index" type. There are some basic things that all index sections need to support to co-exist with the inverted index, but we also allow for the new index sections to support new access methods.

So, this was my approach to extend the existing structures to support new sections in the index (that use some new data-structures appropriate for certain type of data). Something pretty similar should be possible as code is still roughly organized the same way.

If you have other ideas about how to go about this, please share.

from bluge.

mschoch avatar mschoch commented on August 16, 2024

Also, I just remembered, that commit includes an example second index type, it uses a sorted slice and binary search for numeric values. This was not intended to be shipped, but rather to illustrate how a new index section would work.

from bluge.

hengfeiyang avatar hengfeiyang commented on August 16, 2024

Thank you, if have any progress i'll let you know.

from bluge.

hengfeiyang avatar hengfeiyang commented on August 16, 2024

I design like this, What do you think? @mschoch

  1. Change the numeric field behavior like keyword field, store a []byte value of numeric, because we still need search the field like field:value.
  2. Add a type attribute on field, when load index, it create a BKDTree on numeric field for range query.
  3. The query like match query_string field:value just through the default reverted index.
  4. When execute range query will through BKDTree.

As these, it can support range query and don't need create many tokens on numeric field.

from bluge.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.