Comments (1)
FranckPachot
commented on May 23, 2024
Hi
due to the scope of the transaction, at some point Yugabyte needs to have a full copy of the data in memory
It should not. In the SQL layer the memory used is per batch of rows. In the storage layer, it is per RocksDB memtable.
consumes excessive amount of memory,
what is the metric for the graph you show? Can you check mem_tracker_server_Tablets_overhead_PerTablet (for DocDB layer) and allocated_mem_bytes from pg_stat_activity (for the SQL layer)
(e.g. 300mb with around 310 million rows)
What is the size of one row?
Can you share a small sample of the measurement files, to test it.
from yugabyte-db.
Related Issues (20)
- Hidden Tablets Showing as Leaderless in Endpoint View
- [DocDB] Add SCHECK_EC_FORMAT macro
- Add automation for multiple schemas in PG live migration with fall forward
- Add automation for multiple schemas in PG live migration with fall back
- [YSQL] Add config to make ybctid ASC/serial
- [DocDB] Dump memory stacks on high memory usage
- [YSQL] Enable Read committed for new clusters only
- [Tools] yb-ctl fails due to pgrep illegal byte sequence
- [YSQL] clean up YSQL regress test yb_pg_timestamp
- [YSQL] Extra rows printed from index scan with trivial condition
- Integrate NameRegistry to Import Data
- Integrate NameRegistry to Export Data
- Integrate NameRegistry in stats and metadata (YB/meta DB)
- Integrate NameRegistry to Import Data File
- Revisit logic for Init of export-data-from-target;
- [DBZ-PGYB] Validate working with multiple SSL modes in the PG connector
- [DBZ-PGYB] Handle multiple task to support parallelism in PG connector
- [DBZ-PGYB] Port existing tests to PG connector
- [DBZ-PGYB] SQL function to get the list of tablets
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yugabyte-db.