Comments (4)
Hi @delreluca, yes your explanation for what is happening is correct. ManagedRandomAccessFile
is IDisposable
so it's intended to be used with a using
statement and only disposed after it is no longer needed.
This isn't the first time someone has run into this issue though, see #266 for example. So it would probably make sense for ParquetFileReader
to keep a handle to the RandomAccessFile
to avoid this issue. And similarly, ParquetFileWriter
should keep a handle to an OutputStream
when one is used.
from parquetsharp.
This has been fixed on master now, you should still dispose of the ManagedRandomAccessFile
yourself with a using
statement, but ParquetSharp should no longer crash if you don't.
from parquetsharp.
Ah, I missed that issue, sorry. Thanks for confirming my explanation, I can just fix my code with the additional using
then.
I would welcome remembering the handle since I would have expected an IDisposable
to also dispose of whatever it received as a reference in its constructor. (Not sure if that is a written best practice? Just a gut feeling)
from parquetsharp.
I'm not sure it makes sense for ParquetFileReader/Writer
to actually dispose of the ManagedRandomAccessFile
or OutputStream
. That would be a breaking change if someone is relying on being able to reuse them afterwards. But it seems sensible for the reader and writer to just hold a reference to prevent these objects from being garbage collected while in use.
from parquetsharp.
Related Issues (20)
- Broken io exceptions when testing HOT 2
- Exception while reading a column value - ParquetSharp.ParquetException: class parquet::ParquetStatusException (message: 'IOError: Corrupt snappy compressed data.') HOT 1
- isAdjustedToUtc: false HOT 3
- Fails to load ParquetSharpNative.so on CentOS/RedHat HOT 5
- Alpine 3.15 - Unable to load shared library 'ParquetSharpNative' or one of its dependencies. HOT 2
- [CI] vcpkg caching seems broken for macOS
- Type_length for Strings HOT 4
- Support reading string columns with repetition set to required HOT 3
- Strong name for the assemblies in the nuget packages HOT 2
- CI build fails on master HOT 3
- Release ParquetSharp 8.0.0-beta2
- Upgrade to Arrow 9.0.0
- [CI] Fix deprecation warnings HOT 13
- Exception when loading decimal (10,4) column HOT 6
- Cannot decrypt files with pyarrow HOT 2
- Access PageIndex HOT 2
- [Read] Read Decimal LogicalType HOT 5
- RowOriented API fails silently when using internal types in F# HOT 2
- [DOC] How to write nested data struct HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from parquetsharp.