parsyl / parquet Goto Github PK

In Apache Thrift 0.9.3 to 0.13.0, malicious RPC clients could send short messages which would result in a large memory allocation, potentially leading to denial of service.

Fix is in 0.14.0

For now only flat parquet schemas are working correctly.

The code that was used to generate the code for writing values to the input struct needs to be brought back for cases when a field is not repeated.

This library does not support the use of the latest apache thrift library as they have broken their API by introducing new arguments into their exported functions. That event makes it impossible to install this library with go get because the older version is not defaulted to. By adding the specific required version (or by updating this library to be compatible while also still adding these versions to the go.mod) this library will become immediately usable again.

Hi, I have a use-case where I'm writing billions of data entries into a file. I couldn't find any information regarding this in the README, so I wanted to ask whether it's safe to use *ParquetWriter across multiple goroutines to speed up the whole process and if you have any recommendations for doing that. I'm not quite sure but maybe Add could be called concurrently and every once in a while Write should be called depending on the size of RowGroup.

The current implementation misses gzip compression