makl11 / zipper.js Goto Github PK

I want to look into improving the performance of the zip generation.

Some first ideas are:

• Some mostly static Uint8Arrays are potentially created (allocated) many times and create even more intermediate arrays during the creation process. I think there is potential to cut the number of allocations / object creations segnificantly
• Check chunk sizes for input and output streams
• Look into splitting Zipentry.data into several chunks, when it's a Uint8Array
• Look into ReadableStreamBYOBReader, because "it is used for efficient copying from underlying sources [...], such as files" and does zero-copy reading, implying that reading the stream diffrently involves copying data around

Ways I want to profile the performance are:

• Flame Graphs
• Memory Profiling
  • Trace garbage collection
  • Heap profiler

Zipping a 16GB file leads to over 10GB ram usage

I want to be able to use the library in all environments mentioned in the title

• ESM
• CJS
• Browser (bundle crc32 function)
  • ESM
  • Global window object

Atleast the Zipper class should have some documentation

The idea behind this option is to enable resumable downloads of this the generated zip using HTTP range requests. This is supposed to allow users to continue downloading the generated zip after i.e. a network error.

I want to add some tests to make future refactorings easier. Mainly, I want unit tests for the different header and directory sections of the zip format. Also the existing index.test.js is more of a demo than an actual test and thus needs refactoring

• Refactor the current state of index.test.js into an actual test
• Unit test the #encodeNumber method
• Unit test the #dateToDOSTime method
• Unit test the #generateLocalFileHeader method
• Unit test the #generateDataDescriptor method
• Unit test the #generateCentralDirectoryFileHeader method
• Unit test the #generateZip64EndOfCentralDirectoryRecord method
• Unit test the #generateZip64EndOfCentralDirectoryLocator method
• Unit test the #generateEndOfCentralDirectoryRecord method
• Unit test the predictSize method

Using JSDoc was a fun experiment, but I still prefer Typescript
With the correct tsconfig.json, type checking works as expected, no need to migrate to typescript