Giter VIP home page Giter VIP logo

minizip-ng's Introduction

CI Stable Develop
GitHub Actions Stable CMake
Stable Configure
Stable NMake
Develop CMake
Develop Configure
Develop NMake
CodeFactor CodeFactor CodeFactor
OSS-Fuzz Fuzzing Status Fuzzing Status
Codecov codecov codecov

zlib-ng

zlib data compression library for the next generation systems

Maintained by Hans Kristian Rosbach aka Dead2 (zlib-ng àt circlestorm dót org)

Features

  • Zlib compatible API with support for dual-linking
  • Modernized native API based on zlib API for ease of porting
  • Modern C11 syntax and a clean code layout
  • Deflate medium and quick algorithms based on Intel’s zlib fork
  • Support for CPU intrinsics when available
    • Adler32 implementation using SSSE3, AVX2, AVX512, AVX512-VNNI, Neon, VMX & VSX
    • CRC32-B implementation using PCLMULQDQ, VPCLMULQDQ, ACLE, & IBM Z
    • Slide hash implementations using SSE2, AVX2, ARMv6, Neon, VMX & VSX
    • Compare256 implementations using SSE2, AVX2, Neon, POWER9 & RVV
    • Inflate chunk copying using SSE2, SSSE3, AVX, Neon & VSX
    • Support for hardware-accelerated deflate using IBM Z DFLTCC
  • Unaligned memory read/writes and large bit buffer improvements
  • Includes improvements from Cloudflare and Intel forks
  • Configure, CMake, and NMake build system support
  • Comprehensive set of CMake unit tests
  • Code sanitizers, fuzzing, and coverage
  • GitHub Actions continuous integration on Windows, macOS, and Linux
    • Emulated CI for ARM, AARCH64, PPC, PPC64, RISCV, SPARC64, S390x using qemu

History

The motivation for this fork was seeing several 3rd party contributions with new optimizations not getting implemented into the official zlib repository.

Mark Adler has been maintaining zlib for a very long time, and he has done a great job and hopefully he will continue for a long time yet. The idea of zlib-ng is not to replace zlib, but to co-exist as a drop-in replacement with a lower threshold for code change.

zlib has a long history and is incredibly portable, even supporting many systems that predate the Internet.
That is great, but it can complicate further development and maintainability. The zlib code contains many workarounds for really old compilers or to accommodate systems with limitations such as operating in a 16-bit environment.

Many of these workarounds are only maintenance burdens, some of them are pretty huge code-wise. With many workarounds cluttered throughout the code, it makes it harder for new programmers with an idea/interest for zlib to contribute.

I decided to make a fork, merge all the Intel optimizations, some of the Cloudflare optimizations, plus a couple other smaller patches. Then started cleaning out workarounds, various dead code, all contrib and example code.
The result is a better performing and easier to maintain zlib-ng.

A lot of improvements have gone into zlib-ng since its start, and numerous people and companies have contributed both small and big improvements, or valuable testing.

Build

Please read LICENSE.md, it is very simple and very liberal.

There are two ways to build zlib-ng:

Cmake

To build zlib-ng using the cross-platform makefile generator cmake.

cmake .
cmake --build . --config Release
ctest --verbose -C Release

Alternatively, you can use the cmake configuration GUI tool ccmake:

ccmake .

Configure

To build zlib-ng using the bash configure script:

./configure
make
make test

Build Options

CMake configure Description Default
ZLIB_COMPAT --zlib-compat Compile with zlib compatible API OFF
ZLIB_ENABLE_TESTS Build test binaries ON
WITH_GZFILEOP --without-gzfileops Compile with support for gzFile related functions ON
WITH_OPTIM --without-optimizations Build with optimisations ON
WITH_NEW_STRATEGIES --without-new-strategies Use new strategies ON
WITH_NATIVE_INSTRUCTIONS Compiles with full instruction set supported on this host (gcc/clang -march=native) OFF
WITH_RUNTIME_CPU_DETECTION Compiles with runtime CPU detection ON
WITH_SANITIZER Build with sanitizer (memory, address, undefined) OFF
WITH_GTEST Build gtest_zlib ON
WITH_FUZZERS Build test/fuzz OFF
WITH_BENCHMARKS Build test/benchmarks OFF
WITH_MAINTAINER_WARNINGS Build with project maintainer warnings OFF
WITH_CODE_COVERAGE Enable code coverage reporting OFF

Install

WARNING: We do not recommend manually installing unless you really know what you are doing, because this can potentially override the system default zlib library, and any incompatibility or wrong configuration of zlib-ng can make the whole system unusable, requiring recovery or reinstall. If you still want a manual install, we recommend using the /opt/ path prefix.

For Linux distros, an alternative way to use zlib-ng (if compiled in zlib-compat mode) instead of zlib, is through the use of the LD_PRELOAD environment variable. If the program is dynamically linked with zlib, then the program will temporarily attempt to use zlib-ng instead, without risking system-wide instability.

LD_PRELOAD=/opt/zlib-ng/libz.so.1.2.13.zlib-ng /usr/bin/program

Cmake

To install zlib-ng system-wide using cmake:

cmake --build . --target install

Configure

To install zlib-ng system-wide using the configure script:

make install

CPack

After building with cmake, an installation package can be created using cpack. By default a tgz package is created, but you can append -G <format> to each command to generate alternative packages types (TGZ, ZIP, RPM, DEB). To easily create a rpm or deb package, you would use -G RPM or -G DEB respectively.

cd build
cpack --config CPackConfig.cmake
cpack --config CPackSourceConfig.cmake

Vcpkg

Alternatively, you can build and install zlib-ng using the vcpkg dependency manager:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh # "./bootstrap-vcpkg.bat" for powershell
./vcpkg integrate install
./vcpkg install zlib-ng

The zlib-ng port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please create an issue or pull request on the vcpkg repository.

Contributing

Zlib-ng is aiming to be open to contributions, and we would be delighted to receive pull requests on github. Help with testing and reviewing pull requests etc is also very much appreciated.

Please check the Wiki for more info: Contributing

Acknowledgments

Thanks go out to all the people and companies who have taken the time to contribute code reviews, testing and/or patches. Zlib-ng would not have been nearly as good without you.

The deflate format used by zlib was defined by Phil Katz.
The deflate and zlib specifications were written by L. Peter Deutsch.

zlib was originally created by Jean-loup Gailly (compression) and Mark Adler (decompression).

Advanced Build Options

CMake configure Description Default
FORCE_SSE2 --force-sse2 Skip runtime check for SSE2 instructions (Always on for x86_64) OFF (x86)
WITH_AVX2 Build with AVX2 intrinsics ON
WITH_AVX512 Build with AVX512 intrinsics ON
WITH_AVX512VNNI Build with AVX512VNNI intrinsics ON
WITH_SSE2 Build with SSE2 intrinsics ON
WITH_SSSE3 Build with SSSE3 intrinsics ON
WITH_SSE42 Build with SSE42 intrinsics ON
WITH_PCLMULQDQ Build with PCLMULQDQ intrinsics ON
WITH_VPCLMULQDQ --without-vpclmulqdq Build with VPCLMULQDQ intrinsics ON
WITH_ACLE --without-acle Build with ACLE intrinsics ON
WITH_NEON --without-neon Build with NEON intrinsics ON
WITH_ARMV6 --without-armv6 Build with ARMv6 intrinsics ON
WITH_ALTIVEC --without-altivec Build with AltiVec (VMX) intrinsics ON
WITH_POWER8 --without-power8 Build with POWER8 optimisations ON
WITH_RVV Build with RVV intrinsics ON
WITH_CRC32_VX --without-crc32-vx Build with vectorized CRC32 on IBM Z ON
WITH_DFLTCC_DEFLATE --with-dfltcc-deflate Build with DFLTCC intrinsics for compression on IBM Z OFF
WITH_DFLTCC_INFLATE --with-dfltcc-inflate Build with DFLTCC intrinsics for decompression on IBM Z OFF
WITH_UNALIGNED --without-unaligned Allow optimizations that use unaligned reads if safe on current arch ON
WITH_INFLATE_STRICT Build with strict inflate distance checking OFF
WITH_INFLATE_ALLOW_INVALID_DIST Build with zero fill for inflate invalid distances OFF
INSTALL_UTILS Copy minigzip and minideflate during install OFF
ZLIBNG_ENABLE_TESTS Test zlib-ng specific API ON

Related Projects

minizip-ng's People

Contributors

akmistry avatar asmaloney avatar brad0 avatar ccawley2011 avatar coeur avatar dependabot[bot] avatar eiszapfen2000 avatar ffontaine avatar gerbilsoft avatar huven avatar lawadr avatar macgritsch avatar matthew1471 avatar mschmieder avatar nikodelarich avatar nmoinvaz avatar okhowang avatar panovotn avatar peterh avatar pmqs avatar prateek9623 avatar ptc-tgamper avatar rip-nsk avatar rusdevops avatar sergio-nsk avatar stevenchristy avatar sykmschmieder avatar thesamesam avatar vszakats avatar wangp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

minizip-ng's Issues

Unzipping archives in iOS, created on Windows 10 64 bit

I use the zip library 'Zip' from Roy Marmelstein. He has developed that library based on 'minizip'.

Everything works as expected if I create archives with the OS X default compression feature that's integrated in Finder. The archive is getting unzipped successfully by this library.

But as soon as I create an archive on Windows 10 with 7Zip or any other zip program, this archive is recognized as corrupted (bad zip file). If I unzip an archive with Finder, it works. The 'Zip' library throws an exception after calling the function "unzOpenCurrentFile" returned with -103 ('UNZ BADZIPFILE').

I have tried these zip compression algorithms: Deflate, Deflate64, BZip2, LZMA and PPMd.

Is this a known problem, or is a specific compression algorithm necessary?

Thanks in advanced

How to use I/O buffer code in iOS ?

I have a query on how to use the I/O buffer code in place of the existing one. I am writing an iOS app and thinking that it would enhance the performance if I/O memory and buffer code.
Can you please help ?

zipCloseFileInZipRaw64 Xcode "Dead store" warning when HAVE_APPLE_COMPRESSION is not defined

In zip.c, in zipCloseFileInZipRaw64 line 1566, with undefined HAVE_APPLE_COMPRESSION:

        while (err == ZIP_OK)
        {
            uint32_t total_out_before = 0;
            if (zi->ci.stream.avail_out == 0)
            {
                err = zipFlushWriteBuffer(zi);

                zi->ci.stream.avail_out = Z_BUFSIZE;
                zi->ci.stream.next_out = zi->ci.buffered_data;
            }

#ifdef HAVE_APPLE_COMPRESSION
            // ...
#else
            total_out_before = (uint32_t)zi->ci.stream.total_out;
            err = deflate(&zi->ci.stream, Z_FINISH);
            zi->ci.pos_in_buffered_data += (uint16_t)(zi->ci.stream.total_out - total_out_before);
#endif
        }

On the line err = zipFlushWriteBuffer(zi); we are assigning a value to err, but on the line err = deflate(&zi->ci.stream, Z_FINISH); we are re-assigning a new value to err without ever looking at previous value.

Releases and copyright macros

I couldn't find any macro in the source that could provide version check and a copyright notice.

#define MINIZIP_VERSION_MAJOR "1"
#define MINIZIP_VERSION_MINOR "2"
#define MINIZIP_VERSION_PATCH "3"
#define MINIZIP_RELEASE "Minizip " MINIZIP_VERSION_MAJOR "." MINIZIP_VERSION_MINOR "." MINIZIP_VERSION_PATCH

#define MINIZIP_COPYRIGHT MINIZIP_RELEASE "  Copyright (C) 2005-2016 Nathan Moinvaziri"

Something like that would be nice :) I want to give credit to minizip in one of my projects. I don't know if I got the date right (and the versioning system).

Also, currently, I'm not sure if the version in the website is the current version. Using the GitHub tab for releases or using tags would make easier to locate stable releases.

Crash when AES is needed for decryption but no password is passed

It crashes on unzipping aes-protected and NULL is passed as password parameter (you need a way to test if password is needed or not).

I think you can add something like this

#ifdef HAVE_AES
  if (!password && (int)s->cur_file_info.compression_method == AES_METHOD)
    return UNZ_DEFINE_YOUR_OWN_MISSING_PASSWORD_ERROR;
#endif

after

if (unz64local_CheckCurrentFileCoherencyHeader(s, &iSizeVar, &offset_local_extrafield, &size_local_extrafield) != UNZ_OK)
        return UNZ_BADZIPFILE;

Please explain crc_for_crypting

Hello Nathan.

When creating a zip file protected with password, MiniZip APIs (e.g. zipOpenNewFileInZip3_64) require this perplexing crc_for_crypting parameter. The zip.h file just says: "needed for crypting". Following its use in the code provides no particular clues, beside that it's being actively used during encryption.

Could you explain:

  • Why you need this parameter at all (no other encryption APIs that I know of require a CRC, the password should be enough, and data integrity should not be in question).
  • If it is supposed to be a pre-calculated CRC32 of all the files being added to the zip (and if not, what other value is it supposed to be).

As you are aware of, MiniZip is used extensively in Objective-Zip, and I would like to provide a better explanation for this parameter than a simple "needed for crypting".

Thanks in advance.
Keep up the great job!

Zip file corruption after writing less than 4 GB (with 64 bit I/O) on iOS

Hello,

it's again me and it's again an issue on iOS. The sequence to reproduce the problem is the following:

  • create a new file with zipOpen;
  • add a new file to the zip with zipOpenNewFileInZip3 in deflate mode;
  • write uncompressible (e.g.: random) data with zipWriteInFileInZip until you have written 4,294,080,000 bytes (until 4,293,387,000 bytes it works, not sure in the middle);
  • close the file with zipCloseFileInZip.

At this point you will get error -103 (ZIP_BADZIPFILE). The error is produced at line #1808 of zip.c, that is:

        if(uncompressed_size >= 0xffffffff || compressed_size >= 0xffffffff)
        {
            if(zi->ci.pos_zip64extrainfo > 0)
            {
                // Update the size in the ZIP64 extended field.
                // [...]
            }
            else
                err = ZIP_BADZIPFILE; // Caller passed zip64 = 0, so no room for zip64 info -> fatal
        }
        else
            // [...]

It seems that when compiling for iOS the library misconfigures the 64 bit I/O APIs in some way.
Best regards,

Gianluca

Minizip can't open a zip-file created by OS X and bigger than 4 GB

Hi!

I created 2 zip-files by OS X (via context menu -> Compress Files)
photopicker22

One is 6.25 GB and second is 3.78 GB.

Results:

  • Unzipping 6.25 GB zip-file using OSX
  • [!] Unzipping 6.25 GB zip-file using Minizip == ERROR
  • [!] Unzipping 6.25 GB zip-file using standard Windows unzipper == ERROR
  • [!] Unzipping 6.25 GB zip-file using 7-zip (Windows version) == SOME ERRORS. While unzipping, 7-zip says that some files have unsupported compression method.
  • Unzipping 3.78 GB zip-file using OSX
  • Unzipping 3.78 GB zip-file using Mizinip.
  • Unzipping 3.78 GB zip-file using standard Windows unzipper
  • Unzipping 3.78 GB zip-file using 7-zip (Windows version)

Error in unzip.c, started at #1060

if (unz64local_getLong(&s->z_filefunc, s->filestream, &uMagic) != UNZ_OK)
err = UNZ_ERRNO;
else if (uMagic!=LOCALHEADERMAGIC)
err = UNZ_BADZIPFILE;

Its got a wrong 'uMagic' in 'unz64local_getLong' that is not the same as 'LOCALHEADERMAGIC'.

So, looking that OSX is compressing zip-files (which are bigger than 4 GB) by own method that Windows and Minizip can't recognize?

zipWriteInFileInZip hangs when using no compression

Hello,

trying to update Objective-Zip using your version of Minizip, I've noticed that zipWriteInFileInZip hangs when using no compression. The problem seems to be around line #1527 of zip.c, since it loops forever therein.

To reproduce, you can simply substitute your version of Minizip inside Objective-Zip (https://github.com/flyingdolphinstudio/Objective-Zip) and run the test program. If you don't have access to Xcode, here is a brief list of the operations it executes:

  • creates a new file with zipOpen;
  • adds a file with zipOpenNewFileInZip3 (with deflate);
  • writes some data with zipWriteInFileInZip;
  • adds a second file with zipOpenNewFileInZip3 (with no compression);
  • writes some data... and here it hangs.

Thought you might be interested.
Best regards,

Gianluca

How to use minizip to compress a directory?

For example, I want to compress my test directory, which own test2 subdirectory and a.txt file, test2 own b.txt and c.txt, just as followings picture:
image

I found all of file in test diretory, there are /test/a.txt and /test/test2/b.txt and /test/test2/c.txt, and compress them one by one in test.zip file. But when I decompress the test.zip file, the test2 subdirectory is disappear,just as followings picture:
image

Where can I use the wrong? I used the code which as example in minizip,c:
...
...
err = zipOpenNewFileInZip3(zf,filenameinzip,&zi,
NULL,0,NULL,0,NULL /* comment_/,
(opt_compress_level != 0) ? Z_DEFLATED : 0,
opt_compress_level,0,
/_ -MAX_WBITS, DEF_MEM_LEVEL, Z_DEFAULT_STRATEGY, */
-MAX_WBITS, DEF_MEM_LEVEL, Z_DEFAULT_STRATEGY,
password,crcFile);

            if (err != ZIP_OK)
                printf("error in opening %s in zipfile\n",filenameinzip);
            else
            {
                fin = fopen(filenameinzip,"rb");
                if (fin==NULL)
                {
                    err=ZIP_ERRNO;
                    printf("error in opening %s for reading\n",filenameinzip);
                }
            }

            if (err == ZIP_OK)
                do
                {
                    err = ZIP_OK;
                    size_read = (int)fread(buf,1,size_buf,fin);
                    if (size_read < size_buf)
                        if (feof(fin)==0)
                    {
                        printf("error in reading %s\n",filenameinzip);
                        err = ZIP_ERRNO;
                    }

                    if (size_read>0)
                    {
                        err = zipWriteInFileInZip (zf,buf,size_read);
                        if (err<0)
                        {
                            printf("error in writing %s in the zipfile\n",
                                             filenameinzip);
                        }

                    }
                } while ((err == ZIP_OK) && (size_read>0));
              ....
              .....

I called the function three time to compress /test/a.txt , /test/test2/b.txt and /test/test2/c.txt, which pass to the parameter 'filenameinzip',it can work, but the subdirectory test2 is diappear.Please help me!!Thanks!

Provide do_list and do_extract_* as separate library functions

Would you consider providing do_list and do_extract_* as separate library functions? I currently patch minizip just to get these functions without a name conflict with main from miniunz.c.

If such a patch would be accepted I would be happy to provide one.

Implicit conversion loses precision

There are some warnings regarding implicit conversions:

aes/fileenc.c:70:20: Implicit conversion loses integer precision: 'unsigned long' to 'unsigned int'
aes/entropy.c:45:16: Implicit conversion loses integer precision: 'ssize_t' (aka 'long') to 'int'
unzip.c:1269:51: Implicit conversion loses integer precision: 'uLong' (aka 'unsigned long') to 'int'
unzip.c:1269:82: Implicit conversion loses integer precision: 'unsigned long' to 'unsigned int'
unzip.c:1408:71: Implicit conversion loses integer precision: 'uLong' (aka 'unsigned long') to 'unsigned int'

Can't set crc and uncompressed size in case of raw format

As zipCloseFileInZipRaw64 has been removed, there is no more way to set the CRC related to the raw data (zi->ci.raw != 0). Same for the uncompressed size which will then have the value of the "compressed size" when the file is closed.
Can we have a second zipCloseFileInZip method where the CRC and uncompressed size of raw data can be used to skip the related values of the zi structure when closing the file. Thank you

EXC_BAD_ACCESS (code = 1) error when Unzip password protected file with AES method

Hi, I have password protected file with AES method created using this library on iOS, then I unzip it also using this library but sometime the error occurred in the method encr_data of fileenc.h file, and sometime it's extracted successfully. (Ratio is about 2 failed / 10 unzip times)
I think this problem is about memory issue, but I can't find out the reason.

excbadaccess

Here is the password protected file: https://dl.dropboxusercontent.com/u/38691130/han_kaku123-1.zip, (pass: 123)

Thank you so much!

64bit clean

As a suggestion, the code is mixing 64 and 32 bit types in a lot of places, most notably with the CRC calculations which bounce between 32 bit unsigned ints and 64 bit unsigned longs. It's not clear what the intention is then, and you can't cast between these types (uLong and unsigned int) without losing precision. Compiling this for iOS yields a metric ton of assignment errors as 64 bit variables keep getting assigned to 32 bit variables and losing precision. 99% of the time this is going to be OK, but when dealing with encryption and CRC for someone not familiar with the code it's dicey and it's bad practice anyway.

Typical example is here: pos is set up as an unsigned long, 64 bits and then the result is assigned back into a struct which is only holding 32 bits. With some inspection it looks like this can be fixed just by changing the longs to unsigned ints. But this is just one case of many.

static void encr_data(unsigned char data[], unsigned long d_len, fcrypt_ctx cx[1])
{ unsigned long i = 0, pos = cx->encr_pos;

while(i < d_len)
{
    if(pos == AES_BLOCK_SIZE)
    {   unsigned int j = 0;
        /* increment encryption nonce   */
        while(j < 8 && !++cx->nonce[j])
            ++j;
        /* encrypt the nonce to form next xor buffer    */
        aes_encrypt(cx->nonce, cx->encr_bfr, cx->encr_ctx);
        pos = 0;
    }

    data[i++] ^= cx->encr_bfr[pos++];
}

cx->encr_pos = pos;

}

This kind of code then needs to have its types modified to be explicit and no assumptions made that long and int are assignable synonyms (they are not). Where width is a necessary part of the calculation I'd suggest using int32_t, uint32_t, int64_t, uint64_t, etc. CRC calculations are one area where bit with should be explicit and you definitely can't accumulate in one type then assign back and forth to a smaller type and expect your results to be consistent.

UNZ_BADZIPFILE due to incorrect offset of central directory record

A user of my DualBootPatcher project, which uses minizip, let me know that this particular zip file was having trouble being opened: https://drive.google.com/file/d/0B3vBKWqY-uX-QkdCd3BpQ1F1Yzg/view (I apologize for the large file size. I don't know how the file was created, so I couldn't create a smaller sample file.)

I did some digging and it seems that minizip might not be calculating the correct offset of the central directory record. Looking at the output of zipinfo, the central directory should be at 968,265,440 bytes: https://gist.github.com/chenxiaolong/22961396f606118c49d4 However, minizip is seeking to 968,265,516 bytes and failing because the central directory magic doesn't match.

From unz64local_GetCurrentFileInfoInternal in unzip.c, it appears that s->pos_in_central_dir already has the correct offset and s->byte_before_the_zipfile might be throwing the value off.

Any ideas why this might be happening?

Thanks in advance!


I just added a few printf's to see what minizip is doing:

    printf("s->pos_in_central_dir:      %lld\n", s->pos_in_central_dir);
    printf("s->byte_before_the_zipfile: %lld\n", s->byte_before_the_zipfile);
    if (ZSEEK64(s->z_filefunc, s->filestream_with_CD,
            s->pos_in_central_dir + s->byte_before_the_zipfile, ZLIB_FILEFUNC_SEEK_SET) != 0)
        err = UNZ_ERRNO;

    /* Check the magic */
    if (err == UNZ_OK)
    {
        if (unz64local_getLong(&s->z_filefunc, s->filestream_with_CD, &uMagic) != UNZ_OK)
            err = UNZ_ERRNO;
        else if (uMagic != CENTRALHEADERMAGIC) {
            printf("Expected: 0x%08x\n", CENTRALHEADERMAGIC);
            printf("Actual:   0x%08x\n", uMagic);
            err = UNZ_BADZIPFILE;
        }
    }

Output:

s->pos_in_central_dir:      968265440
s->byte_before_the_zipfile: 76
Expected: 0x02014b50
Actual:   0xdf08077f

encrypted ZIPs unzReadCurrentFile -3

When ever I try a password protected ZIP I only get -3 from unzReadCurrentFile.

Using macOS Sierra zip command to create encrypted zip file:
zip -re ./PasswordTest.zip ./PasswordTest

This file works with the build in Archive Utility and third party tools like The Unarchiver.

See attached sample. Password: passw0rd

PasswordArchive.zip

unzOpenInternal unused parameter leading to question...

In unzip.c the prototype for unzOpenInternal is:

local unzFile unzOpenInternal(const void *path, zlib_filefunc64_32_def* pzlib_filefunc64_32_def, int is64bitOpenFunction)

The parameter is64bitOpenFunction is unused in the function.

Looks like this is the code where that would have been used:

    if (pzlib_filefunc64_32_def == NULL)
        fill_fopen64_filefunc(&us.z_filefunc.zfile_func64);
    else
        us.z_filefunc = *pzlib_filefunc64_32_def;

But then unzOpenInternal is called like this:

extern unzFile ZEXPORT unzOpen(const char *path)
{
    return unzOpenInternal(path, NULL, 0);
}

extern unzFile ZEXPORT unzOpen64(const void *path)
{
    return unzOpenInternal(path, NULL, 1);
}

Clearly expecting different results.

Anyone have any insight into what this was used for and what should happen here? Without really knowing the code I can only guess at the intent.

Add xcode project

This issue is for adding an xcode project to the source code for people on Apple devices.

unzseek64() compression_method vs. compression level

I am not sure how unzseek() is supposed to work. Line 1897 contains the following code:

if (pfile_in_zip_read_info->compression_method != 0) return UNZ_ERRNO;

compression_method is always Z_DEFLATED (8) - althoug it seems it is sometimes used as compression level in the code? I think the idea was to check if compression level was 0 but I am not sure if this info is avaiable. How is this supposed to work?

not compiling with AES on windows

in CMakeLists:

set_target_properties(aes minizip
   PROPERTIES
   COMPILE_DEFINITIONS "O -DHAVE_AES")

does not work on windows with Visual Studio 2015. Changed to

add_definitions(-DHAVE_AES)
and it compiles fine.

Also added
add_definitions(-D_CRT_SECURE_NO_DEPRECATE)
to
if(WIN32)

Unable to store files into zip file without -j option

On running with options as specified in usage(without -j), minizip.exe generates an empty zip file whereas if -j option is added it creates a zip file with the file in it. miniunz.exe works fine without any issues.

Add support for zip streaming

As far as I can tell there is no support for compression streaming (i.e. using the library without ftell) because after a file is compressed it will fseek back to write the original size, compressed size and CRC32 for the last added file. However, it is supported by the zip standard to write this three values after the last file for especially this purpose.

I would also be willing to supply a pull request with an implementation if someone can propose the necessary change to the interface.

"Memory Error" warning when using AES

This is an Xcode 8 buildtime warning.
screen shot 2017-06-05 at 14 03 21

It says that memory allocated on pfile_in_zip_read_info = (file_in_zip64_read_info_s*)ALLOC(sizeof(file_in_zip64_read_info_s)); is not freed in case password == NULL.

Implementation needs support for "AE-2" encrypted zip

Originally I misunderstood the AES WinZip specification and posted suggestion #11 thinking that the current implementation is 'vendor-specific.' However, after some additional research I believe the issue is that current Zip programs are not using AE-1 due to it having some security weakness. The unzip code can be modified to unzip AE-2 encrypted files just by modifying the version test I pointed out in the previous issue ticket. However, this change alone does not add a test for the authentication code to verify that the contents have not been modified since encrypted.

unzLocateFile randomly not working

I have a problem with my Swift minizip wrapper (creating inputStream from zip to access a particular ressource inside the archive)

It's working, but it crashes around 10% of time.

Sometimes the unzLocateFile() calls fails, and return the UNZ_END_OF_LIST_OF_FILE error.
It's caused by the sub calls to unzGoToNextFile2().

(unzip.c line 1678) the error is returned here. That mean that one time out of ten the minizip lib can't find the file in my archive? (but the previous times it found it, same file, same archive)

Am I doing something wrong?

Unzip AES support intentionally restricted to a specific vendor id?

Nice work putting all this together. I encountered only one small issue trying to use this library. My 7zip AES encrypted zip file would not unzip properly. I traced the problem down to unz64local_GetCurrentFileInfoInternal.

Lines:

else if (headerid == 0x9901)
{
      /* Subtract size of AES field, since AES is handled internally */
      file_info.size_file_extra_internal += 2 + 2 + datasize;

      /* Verify version info */
      if (unz64local_getShort(&s->z_filefunc, s->filestream_with_CD, &uL) != UNZ_OK)
            err = UNZ_ERRNO;
      if (uL != 1)   // <---- this makes the unzip AES vendor specific. 7zip is putting 2 here.
            err = UNZ_ERRNO;
      if (unz64local_getByte(&s->z_filefunc, s->filestream_with_CD, &uL) != UNZ_OK)

The code tests if vendor id != 1 and if so it returns an error. Just my opinion, but I think the test should be removed entirely. Otherwise it should be noted that 7zip is compatible and uses 2 so the test could be changed to be uL != 1 && ul != 2.

If safety is the concern, one is probably better off making sure that file_info_internal.aes_encryption_mode and file_info_internal.aes_compression_method are being set to valid values rather than checking the vendor id. Just my opinion though.

64 bits vs 32 bits

Hi, I'm using your lib in my game engine, and yesterday i have tried to unzip a file generated on macosx (64) on windows 10 (64), but for my surprise it can't open even in winrar, and the file generated on windows doesn't open on linux (64) or macosx (64)...
So i don't know if its a os problem or architecture.
If you guys wanna test it, here are the links:
https://www.dropbox.com/s/l1h1zz5n9ivkdup/data.magic3d
https://www.dropbox.com/s/5s4dlp0qvmytwfd/data.magic3d_windows

My project is https://github.com/magic-tech/Magic3D.git and it is compiled with QT 5.5.1 using mingw32
Thanks in advance.

PKWARE ZIP encryption issue

When zlib/minizip are built on 64-bit Linux, ZIP archives with password/encryption cannot be unzipped (always getting "invalid password")
The issue does not occur when zlib/minizip is built in 32 bit.

I found the solution to be:

ioapi.h:62

ifdef HAS_STDINT_H

which should be

ifdef HAVE_STDINT_H

I'm new to GitHub, so pls make the change and commit, as I don't want to "mess up things".

thanks,

Joerg

zipCloseFileInZip is setting uncompressed size to 0 (related to issue #106)

In zip.c, zipCloseFileInZipRaw64, in case of non raw format, the following 2 lines (#1628 - 1629)
crc32 = zi->ci.crc32;
uncompressed_size = zi->ci.total_uncompressed;
shall be moved a few lines down, before the /* Write data descriptor */ section to ensure stream compression has ended to allow proper reading of the uncompressed size value.

Unzip with UNZ_BADZIPFILE error (compression_method = 99)

Hi, I received a zip file and don't know exactly which OS, as well as which tool or compression method it's zipped by. I've used your lib with latest version to unzip it but it throw UNZ_BADZIPFILE error with the compression_method = 99. (I've debugged and realized that other zip file's compression_method is 8 or something not 99)
(I'm really sorry that I can't upload that file due to its privacy)

compressionmethoderror

The above code snippet is from unz64local_CheckCurrentFileCoherencyHeader method (unzip.c file).
Thank you very much!

Warning in ioapi_mem.h on iOS 9

ioapi_mem.h

Line 104
newmemsize = mem->size;

Warning: Implicit conversion loses integer precision: 'uLong' (aka 'unsigned long') to 'int'

Compiling issue

When I try to use minizip in my project, I got linking error below. Does minizip depend on any external library and where can I get the library (if it depend on it)?

unzip.obj: -1 : error: LNK2019: unresolved external symbol inflate referenced in function unzReadCurrentFile

unzip.obj: -1 : error: LNK2019: unresolved external symbol inflateEnd referenced in function unzCloseCurrentFile

unzip.obj: -1 : error: LNK2019: unresolved external symbol crc32 referenced in function unzReadCurrentFile

unzip.obj: -1 : error: LNK2019: unresolved external symbol inflateInit2_ referenced in function unzOpenCurrentFile3

unzip.obj: -1 : error: LNK2019: unresolved external symbol get_crc_table referenced in function unzOpenCurrentFile3

unzReadCurrentFile crashes if pass NULL as parameter password in method unzOpenCurrentFilePassword

Hi!

Crash occurs in line 1408 because parameter s->pcrc_32_tab used without initialization.
Initialization of this parameter occurs in the method unzOpenCurrentFile3 only if password != NULL.
In original minizip 1.1 for this purpose used flag s->encrypted, but it can be fixed without this flag just if
set s->pcrc_32_tab=0 in unzOpenCurrentFile3 and check it before use s->pcrc_32_tab in line 1395

if ( s->pcrc_32_tab && ( (s->cur_file_info.flag & 1) != 0 ) ) { ... }

Passing incompatible pointer types

There are some warning regarding passing incompatible pointer types:

unzip.c:898:79: Incompatible pointer types passing 'uLong *' (aka 'unsigned long *') to parameter of type 'int *'
unzip.c:902:79: Incompatible pointer types passing 'uLong *' (aka 'unsigned long *') to parameter of type 'int *'
unzip.c:907:79: Incompatible pointer types passing 'uLong *' (aka 'unsigned long *') to parameter of type 'int *'
unzip.c:1269:72: Passing 'const char *' to parameter of type 'const unsigned char *' converts between pointers to integer types with different sign

Zip file created with Password on iOS can't unzip on Windows

Hi, I'm using your lib in my iOS project. My problem is the zip file that is created with password on iOS 10.0 (Both Simulator and Real iPhone) can't unzip on Windows. I've tried many unzip tools (WinRAR, 7Zip...) but all failed with a message "Wrong password". (The problem is only occurred on my password protected zip file, if password is null, it is perfectly unzipped on Windows)

I've looked for answers from Closed Issues and saw a post that's look like my problem (#45), but it's not solved in my case.

Here is the zipped file https://dl.dropboxusercontent.com/u/38691130/image_pass_123.zip
Password is "123"

Thankyou so much!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.