Comments (2)
Hello, I'm a newbie in the community and I'd like to try this issue. But I want some help. Now, I seeBlockVersion::SaveToFile is write to a file not catalog. How can I understand "store block version information into a file". And where can I got some " block column data solution" refer. Thanks
from infinity.
Your understanding is correct, the current version file can now be stored on the disk. However, this file is currently considered as part of the catalog. We can see that the BlockEntry:: Deserialize function directly reads the version file.
We have conducted some tests before, and when a large amount of data is inserted into infinity, this version file will occupy a lot of memory in memory.
Perhaps the initial description of this issue led you to misunderstandings. The purpose of this issue is to manage version files like block column files using a buffer manager. Use pointers like buffer objects for reading and writing. Behind the Buffer Manager is a component that can manage global objects within the database
from infinity.
Related Issues (20)
- [Feature Request]: HOT 3
- [Feature Request]: Support default value when INSERTION and IMPORT json/jsonl file. HOT 1
- [Bug]: Corruption happens if empty data is used for inverted indexing
- [Bug]: test_insert_big_embedding_various_type failed HOT 1
- [Feature Request]: Building for ARM64 CPUs
- [Bug]: http api: insert vector with type of integer will lead to bad resp HOT 1
- [Bug]: http_api,update vector leads bad resp
- [Bug]: Decrease row count exceed actual row count@src/storage/meta/entry/segment_entry.cppm:184 HOT 1
- [Bug]: http api,the import option header is actually always false HOT 1
- [Bug]: http api: can not import tmp_20240116.csv
- [Feature Request]: Improve delta op performance
- [Question]: Does fulltext index support Chinese? HOT 2
- [Bug]: Cannot find full checkpoint file in shutdown process. HOT 1
- [Bug]: Checkpoint write uncommitted row count.
- [Bug]: insert enwiki datasets failed HOT 2
- [Bug]: Infinity can not start up correctly with segment error HOT 1
- [Feature Request]: Integrate Japanese morphological analyzer into Infinity
- [Bug]: Cleanup is too aggressive HOT 1
- [Feature Request]: Improve the performance of block-max wand/maxscore for disjunctive scoring
- API CALLS HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from infinity.