Comments (3)
TimJMartin
commented on June 25, 2024
2
@szarnyasg - that has fixed it. Thank you so much for pointing me towards that setting.
Reading the docs "When importing or exporting data sets that are much larger than the available memory, out of memory errors may occur." Because my dataset (12Gb uncompressed CSV) is much smaller than my memory (32Gb) I wonder if this is happening more often to people.
from duckdb.
Mytherin
commented on June 25, 2024
1
There's a fix for this on the latest main - see #10756
from duckdb.
szarnyasg
commented on June 25, 2024
Hi @TimJMartin, did you try running the workload with the following setting?
SET preserve_insertion_order = false;https://duckdb.org/docs/guides/performance/how_to_tune_workloads#the-preserve_insertion_order-option
from duckdb.
Related Issues (20)
- python : query parameterization fails for IN clause HOT 1
- Tests fail because of missing version: is git required? HOT 2
- ParamsToString() in GET table filters missing apostrophes in VARCHAR fields HOT 5
- 0.10.2 introduces new Error: Binder Error: Cannot compare values of type VARCHAR and type DATE - an explicit cast is required HOT 3
- CSV rejects table error_message is the same for each line despite column having different values HOT 1
- Documentation for COPY TO: Please mention there is no need for overwrite guard code HOT 4
- FLOAT/Integer gives FLOAT, when it should be a Double HOT 14
- read_csv ignore_errors does not work as expected HOT 2
- Duckdb (Python) 0.9.1 + 0.10.2 fails to read empty parquet files HOT 18
- 2 tests fail
- ORDER BY is not implemented for window functions HOT 2
- Can't replace single quote character HOT 3
- Out of memory on exploding join HOT 3
- Executed query is being run again HOT 2
- CREATE UNIQUE INDEX IF NOT EXISTS doesn't work any more HOT 1
- Fatal error (crash) when flattening fixed length arrays using the flatten() function HOT 5
- Incorrect TIMETZ comparisons
- Cannot `insert into ... with foo as materialized (...) select * from foo` in 0.10.2 HOT 1
- Magic numbers in `serializer.WriteProperty` and `deserializer.ReadProperty`
- concat_ws and list aggregate HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from duckdb.