Comments (9)
Difficult to say without the zip file itself, might be a fixable bug, might be that the zip file was produced in a way that makes it impossible to process by streaming.
From README:
Please note that the zip file format isn't really meant to be processed by streaming, though this library should succeed in most cases, if you do have complete zip file available, you should consider using other libraries which read zip files from the end - as originally intended (for example yauzl or decompress-zip).
from unzip-stream.
from unzip-stream.
If you have control over the original zip file you can just try to compress it with a different program - for example the Mac archive utility should produce zips that can be always processed with a streaming unzip.
from unzip-stream.
from unzip-stream.
Fair enough, if you have an example zip that can be shared, I can have a look if it's something fixable.
from unzip-stream.
Hi,
I got this same issue, if I downlaod the zip file and uncompress it locally its works without any problem.
But if I try to stream the file over network and uncompress it using pipe it throws this error.
var readStream = sftp.createReadStream(
/Import/${fileNameArray[0]}); readStream.pipe(unzip.Parse()) .pipe(stream.Transform({ objectMode: true, transform: (entry, encoding, callback) => { console.log(
Inside transform callback....); const outputPath = path.join(dir, entry.path); entry.pipe(fs.createWriteStream(outputPath)) .on('finish', () => { console.log(
Done writing file ${entry.path}); }) .on('error', err => { console.log(
Error writing stream ${JSON.stringify(err)}); }); } }));
Erro thrown is as shown below:
Error: Stream finished in an invalid state, uncompression failed
at UnzipStream._flush (/home/user/Nodejs/fileDownload/node_modules/unzip-stream/lib/unzip-stream.js:734:19)
at UnzipStream.prefinish (_stream_transform.js:137:10)
at emitNone (events.js:106:13)
at UnzipStream.emit (events.js:208:7)
at prefinish (_stream_writable.js:603:14)
at finishMaybe (_stream_writable.js:611:5)
at endWritable (_stream_writable.js:622:3)
at UnzipStream.Writable.end (_stream_writable.js:573:5)
at ParserStream._flush (/home/user/Nodejs/fileDownload/node_modules/unzip-stream/lib/parser-stream.js:33:22)
at ParserStream.prefinish (_stream_transform.js:137:10)
from unzip-stream.
from unzip-stream.
I got same error.
Before unzip, I print out the file size. The file size is correct.
when the program second time unzip the file. It runs successfully.
(node:25) UnhandledPromiseRejectionWarning: Error: Stream finished in an invalid state, uncompression failed
appstore-api-server_1_450bccf02a4c | at UnzipStream._flush (/node_modules/unzip-stream/lib/unzip-stream.js:734:19)
appstore-api-server_1_450bccf02a4c | at UnzipStream.prefinish (_stream_transform.js:137:10)
appstore-api-server_1_450bccf02a4c | at emitNone (events.js:106:13)
appstore-api-server_1_450bccf02a4c | at UnzipStream.emit (events.js:208:7)
appstore-api-server_1_450bccf02a4c | at prefinish (_stream_writable.js:602:14)
appstore-api-server_1_450bccf02a4c | at finishMaybe (_stream_writable.js:610:5)
appstore-api-server_1_450bccf02a4c | at endWritable (_stream_writable.js:621:3)
appstore-api-server_1_450bccf02a4c | at UnzipStream.Writable.end (_stream_writable.js:572:5)
appstore-api-server_1_450bccf02a4c | at Extract._flush (/node_modules/unzip-stream/lib/extract.js:41:22)
appstore-api-server_1_450bccf02a4c | at Extract.prefinish (_stream_transform.js:137:10)
from unzip-stream.
Getting the same error occasionally:
Error: Stream finished in an invalid state, uncompression failed
at UnzipStream._flush (/usr/src/app/node_modules/unzip-stream/lib/unzip-stream.js:734:19)
at UnzipStream.prefinish (_stream_transform.js:142:10)
at UnzipStream.emit (events.js:314:20)
at prefinish (_stream_writable.js:628:14)
at finishMaybe (_stream_writable.js:636:5)
at endWritable (_stream_writable.js:656:3)
at UnzipStream.Writable.end (_stream_writable.js:595:5)
at ParserStream._flush (/usr/src/app/node_modules/unzip-stream/lib/parser-stream.js:33:22)
at ParserStream.prefinish (_stream_transform.js:142:10)
at ParserStream.emit (events.js:314:20)
For some context, I am piping a read stream into unzip-stream
from a remote file (fetched using gaxios using responseType: 'stream'
). Mostly this works fine. When it doesn't work (with the above error) it is always when the node on which this is running is also fetching and processing lots of other files (in other node apps running on the same node)... so resources are somewhat limited. My guess is that the remote fetch operation times out after some time, so that the stream into unzip-stream
ends prematurely. Is there any way to avoid this?
Incidentally, I get an equivalent error under the same circumstances with related node-unzipper
... see here... I came to this library in the hope that it performed better in these circumstances but it looks like it fails in a similar way.
from unzip-stream.
Related Issues (20)
- After I unziped, lost a lot of files. HOT 1
- unzip.Extract() close event is not called HOT 2
- How many entries? HOT 1
- Close getting fired before finishing Transform on all files HOT 3
- EMFILE error happened when the zip file is too big HOT 1
- Consider removing invalid test data HOT 1
- Publish new version to npm registry HOT 1
- Should check crc32s against the entry data
- Extract Method cannot allow passing decodeString property HOT 1
- How to catch and resolve error "RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 5. Received 9"
- when are you planning to make a release to npm? HOT 1
- Support passing file comments through for each entry HOT 4
- Extract() loses file permissions from zip HOT 10
- new Buffer() is deprecated
- Total number of entries? HOT 2
- Uncatchable errors HOT 2
- Unsupported zip version HOT 4
- 10 year old dependency
- LastModified dateTime for entries
- Vulnerable dependency in binary- 0.3.0 package.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from unzip-stream.