Please provide archives for releases about rules_android HOT 9 OPEN

Per that thread (emphasis mine):

In general we don't guarantee that the checksums of zip downloads will stay the same, they do for downloads of releases though.

Given this should the README for this repo update the URL in its example to the URL available from the release page rather than the download URL ?

Download URL: https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip
⬇️
URL copied from the releases page: https://github.com/bazelbuild/rules_android/archive/refs/tags/v0.1.1.zip

For me at least the former is still producing inconsistent output and the latter is consistent

from rules_android.

Using https://mirror.bazel.build/github.com/bazelbuild/rules_android/archive/v0.1.1.zip for now I suppose

from rules_android.

Github made a change that caused the zips to change, they've since rolled it back:
https://twitter.com/tgummerer/status/1488493440103030787

Let's leave this issue open though in case Github decides against making guarantees around these zip files.

from rules_android.

They all seem to match for me now:

$ curl -sL https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip | sha256sum
cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806  -
$ curl -sL https://github.com/bazelbuild/rules_android/archive/refs/tags/v0.1.1.zip | sha256sum
cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806  -

Perhaps the rollback is gradual?

from rules_android.

Worked with @ahumesky to get to the bottom of what might be going on right now (and a huge thank you to him for helping me with this)

As of right now it appears that the link https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip returns an inconsistent result depending on if it is fetched using http/2 or http/1.1

$ curl -sL --http2 https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip | sha256sum
cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806  -
$ curl -sL --http1.1 https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip | sha256sum
7cb2b2e11aafe08b2ccb6eeb11f21f52c45e1ad1cdd0132ddf7bde8462bd9e20  -

this explains why wget and curl were giving different results in #42 (wget only supports http/1.1).

@ahumesky discovered that the difference between the http/1.1 and http2 versions was the timestamps between the files inside the zip archives.

Per https://twitter.com/tgummerer/status/1488493440103030787 the hashes should be matching but there still appears to be a lingering bug for http/1.1 responses, presumably the repackaged zip files are cached incorrectly.

Updating the link to instead be https://github.com/bazelbuild/rules_android/archive/refs/tags/v0.1.1.zip (including refs/tags) in the URL currently returns a consistent hash across http/1.1 and http/2.0

it is hard to say if this is by design (archive/$tag: unstable, archive/refs/tags/$tag: stable) or if one of the values just happens to be cached incorrectly.
This behavior can also be seen in emscripten-core/emscripten#16164 where the archive/refs/tags link is stable but the archive/ link is not

from rules_android.

There's some discussion in the Bazel slack community about this as well. https://bazelbuild.slack.com/archives/CA31HN1T3/p1643720300781339

from rules_android.

Using one of the examples from that Slack thread I can see that it's not only http/2 causing the issue as one of the examples there does produce differing files despite not forcing http/1.1

$ (while true; do curl -sL https://github.com/yaml/pyyaml/archive/5.1.2.zip | sha256sum; sleep 0.5; done;) | uniq
e9df8412ddabc9c21b4437ee138875b95ebb32c25f07f962439e16005152e00e  -
ba59d7e97eb131d8f8f52d19cb124bb67772f4c7f4d14cb2919deb885ef8c572  -
<repeats>

consistent with emscripten and this repo, replacing archive with archive/refs/tags produces consistent results but I don't know if that's because maybe there's no cached CDN hit for that specific URL from the time that the feature flag that repackaged zip files was active.

If I make multiple requests over time to https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip using http/1.1, I can see occasionally the original file, cd06d15... is returned. http/2 requests only seem to return only cd06d15... as far as I can tell

At this point I believe the following are true:

• GitHub did not fully invalidate the cache for all of the zip files updated by the rolled-back feature flag
• GitHub partitions its http/1.1 and http/2 caches separately
  • This means sometimes, but not always, forcing http/2 fixes the issue

Unproven (but not disproven afaik, I'm hoping this is true):

• archive/refs/tags/$tag urls appear to have been unaffected by this change

Hopefully this sorts itself out tomorrow, I raised a bug against GitHub, hopefully we can get some sort of guarantee around archive consistency.

from rules_android.

Github made a change that caused the zips to change, they've since rolled it back: https://twitter.com/tgummerer/status/1488493440103030787

Let's leave this issue open though in case Github decides against making guarantees around these zip files.

@ahumesky both that tweet you linked to and the issue I linked to mention that GitHub does not guarantee these are stable, the only way is to download the tarball at release creation and upload it as an asset. Even though this time the archive/ref links didn't change GitHub is reserving the right too

https://twitter.com/tgummerer/status/1488493481874055173?s=20&t=447rsX6z1sn19TUvqSdVzw
Homebrew/homebrew-core#18044 (comment)

from rules_android.

Related: bazelbuild/bazel#15128 (comment)

from rules_android.