Giter VIP home page Giter VIP logo

deterministic-zip's Introduction

Hi there, I`m Timo! ๐Ÿ‘‹

Techie by heart. Full-Stack Web Development, Software Architect and DevOps.

๐Ÿ’ฌ Ask me about anything here

More about me?

Fancy stats go here

Name Stat
General Timo's github stats
Languages
Contribution Streak
Trophies trophy

General ideology across projects

When I create something that could be helpful for other people, it's shared publicly with the world (you!)

No guarantees when it comes to response times

Since this is something completely done in my freetime, response time (e.g. when I am on vacation etc.) will be sometimes slower. Nevertheless, I always try to deliver fast and respond quickly to questions/issues/feedback etc.

Stale projects wont get deleted

Whenever a project is discontinued, for whatever reason it will not be deleted but rather archived. This way you can have confidence that projects maintained by me will not just disappear. Of course also feel free to fork it anyway.

Feature requests

Feature requests will be carefully evaluated by me, and I reserve the right to decline requests that either not comply with the overall goal of a project or my opinion.

Fun (avatar) stuff

Octocat
Octocat SouthPark Avatar Current gravatar

Wanna buy me a coffee?

Buy Me A Coffee

deterministic-zip's People

Contributors

nathanklick avatar renovate[bot] avatar timo-reymann avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

deterministic-zip's Issues

Make output less verbose by default

Description

By default, deterministic-zip outputs information about all files added, this clutters output.

It would be nicer to display this information only in verbose mode, also making output more consistent and easier.

Proposal for new outputs in verbose mode:

Action Sample output
Added to file set + {file-name} added to file set by '{feature name}'
Removed from file set by filter - {file-name} removed from file set by '{feature name}'
Feature loaded โ—‰ {Feature feature-name is active}
Feature not loaded โ—ฏ {Feature feature-name is not active}

Regular debug outputs start with debug:, directly added in log tool.

If it makes output easier to read at first glance, colored output, if supported by the target system, would be a benefit.

Code changes

  • Source files are not modifiable from outside
  • Source files can be changed with Add/Remove method
  • Add debug prefix for debug outputs, making it easy to distinguish

Benefits of new output format

Allows effortless troubleshooting if, e.g., the final zip in CI looks weird and should be more straightforward to debug with clear and consistent output.

References

No response

I would like to see docker images and github actions

Community Note

  • Please vote on this issue by adding a ๐Ÿ‘ reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

I like this tool, because it solves a problem I have when building lambda functions for aws and only want to deploy them if anything changes - so deterministic builds are very helpful.

But it would be great, if there are docker images and github actions I can use :-)

I may be able to implement this, but are you interested in seeing this project?

Create GitHub action

Description

Currently there is already an CircleCI Orb for deterministic-zip, so it would be nice to also provide a GitHub Action for users that use that for their deployments etc.

References

#2

[Bug]: Ignore own file for zipping

What happened?

When you want to zip the directory that also contains the target zip file, deterministic-zip will try to add the currently open file, which results in an infinite growing file.

OS

Ubuntu

Problem description

Only possible if the zip file will be in the same file, there are no other things required

Expected behaviour

When deterministic-zip encounters itself in the file list, it should be ignored

Actual behaviour

The application tries to add the zip file which is growing, effectively, resulting in an endless loop.

Steps to Reproduce

  1. Create a folder with some files
  2. Try to zip the current folder, e.g., deterministic-zip -r test.zip .
  3. The tool will try to add the file until the process is killed

Important Factoids

No response

References

No response

I read and aggree to the contribution guidelines

  • I read and aggree to the contribution guidelines

Include directory entries in the zip file

Community Note

  • Please vote on this issue by adding a ๐Ÿ‘ reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Due to the limitations of some zip implementations, it is sometimes necessary to include directory entries in the zip file. Additionally, in certain use cases an empty directory should be created when the zip file is extracted.

References

no such file or directory (parity issue with regular zip)

Description

When zipping with regular zip, you can name files/directories that don't exist, and it will simply print a message (that can be suppressed with -q)

On deterministic-zip, if the file/directory doesn't exist it prints a message even with -q and won't zip at all.

References

No response

Add tests unzipping file content

Community Note

  • Please vote on this issue by adding a ๐Ÿ‘ reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Verify that the zipped files are extractable to prevent bugs like the one with the directories being created as files.

Add an option to recreate the archive if needed with -FS

Description

Currently using -FS / --filesync appends the newly created files to the end of the archive which makes it non deterministic, having an extra option that deletes the archive and recreates it if needed would be nice to have.

References

No response

[Bug]: Latest v1 release is broken

What happened?

The binary downloaded by installer script is not found in the latest v1 release.

OS

Ubuntu

Problem description

The following link is generated by the installer script, which points to a binary file that does not exist.

curl -LsS https://github.com/timo-reymann/deterministic-zip/releases/download/v1/deterministic-zip_linux-amd64

Output:
Not Found

Expected behaviour

Correct binary to be downloaded.

Actual behaviour

Link to binary is broken.

Steps to Reproduce

curl -LsS https://github.com/timo-reymann/deterministic-zip/releases/download/v1/deterministic-zip_linux-amd64

Important Factoids

No response

References

No response

I read and aggree to the contribution guidelines

  • I read and aggree to the contribution guidelines

[Bug]: bad zipfile offset

What happened?

I'm zipping a folder using: deterministic-zip -r ../logs-extension-api.zip . -x "*.rush/*" -x "*__tests__/*" -x "*.log" -v.
The zip is generated without issue, but when I do: unzip ./logs-extension-api.zip it says:

Archive:  ./logs-extension-api.zip
.rush/temp/operation/zip/state.json:  mismatching "local" filename (extensions-api.js),
         continuing with "central" filename version
  inflating: .rush/temp/operation/zip/state.json  
  error:  invalid compressed data to inflate
file #2:  bad zipfile offset (local header sig):  148
file #3:  bad zipfile offset (local header sig):  1028
file #4:  bad zipfile offset (local header sig):  4378
file #5:  bad zipfile offset (local header sig):  5210
file #6:  bad zipfile offset (local header sig):  5844
file #7:  bad zipfile offset (local header sig):  6663
file #8:  bad zipfile offset (local header sig):  7399
file #9:  bad zipfile offset (local header sig):  8076
file #10:  bad zipfile offset (local header sig):  8754
file #11:  bad zipfile offset (local header sig):  9311
file #12:  bad zipfile offset (local header sig):  9708
file #13:  bad zipfile offset (local header sig):  9846
file #14:  bad zipfile offset (local header sig):  10430
file #15:  bad zipfile offset (local header sig):  11177
file #16:  bad zipfile offset (local header sig):  22049
file #17:  bad zipfile offset (local header sig):  22374
file #18:  bad zipfile offset (local header sig):  23003
file #19:  bad zipfile offset (local header sig):  23383

I tried to remove the -x options and it works perfectly.

OS

Darwin ARM

Problem description

I think the repetition of -x options make a problem in the zip file

Expected behaviour

Should unzip without issue

Actual behaviour

I got this error while unzip:

Archive:  ./logs-extension-api.zip
.rush/temp/operation/zip/state.json:  mismatching "local" filename (extensions-api.js),
         continuing with "central" filename version
  inflating: .rush/temp/operation/zip/state.json  
  error:  invalid compressed data to inflate
file #2:  bad zipfile offset (local header sig):  148
file #3:  bad zipfile offset (local header sig):  1028
file #4:  bad zipfile offset (local header sig):  4378
file #5:  bad zipfile offset (local header sig):  5210
file #6:  bad zipfile offset (local header sig):  5844
file #7:  bad zipfile offset (local header sig):  6663
file #8:  bad zipfile offset (local header sig):  7399
file #9:  bad zipfile offset (local header sig):  8076
file #10:  bad zipfile offset (local header sig):  8754
file #11:  bad zipfile offset (local header sig):  9311
file #12:  bad zipfile offset (local header sig):  9708
file #13:  bad zipfile offset (local header sig):  9846
file #14:  bad zipfile offset (local header sig):  10430
file #15:  bad zipfile offset (local header sig):  11177
file #16:  bad zipfile offset (local header sig):  22049
file #17:  bad zipfile offset (local header sig):  22374
file #18:  bad zipfile offset (local header sig):  23003
file #19:  bad zipfile offset (local header sig):  23383

Steps to Reproduce

try to use the command with one or multiple -x flag

Important Factoids

No response

References

No response

I read and aggree to the contribution guidelines

  • I read and aggree to the contribution guidelines

argument list too long

Community Note

  • Please vote on this issue by adding a ๐Ÿ‘ reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment

Used versions

  • Host OS: Darwin

Problem description

Trying to zip some files, like this deterministic-zip -D package.zip node_modules/**/*

Expected Behavior

Files are added to the zip

Actual Behavior

I get this error
zsh: argument list too long: deterministic-zip

Steps to Reproduce

Important Factoids

References

  • #0000

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

This repository currently has no open or pending branches.

Detected dependencies

circleci
.circleci/config.yml
  • github-cli 2.3.0
  • docker 2.6.0
  • codecov 4.1.0
  • cimg/go 1.22
  • cimg/python 3.12
dockerfile
Dockerfile
gomod
go.mod
  • go 1.19
  • github.com/gobwas/glob v0.2.3
  • github.com/spf13/pflag v1.0.5

  • Check this box to trigger a request for Renovate to run again on this repository

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.