Giter VIP home page Giter VIP logo

repro-sources-list.sh's Introduction

repro-sources-list.sh

repro-sources-list.sh configures /etc/apt/sources.list and similar files for installing packages from a snapshot to help Reproducible Builds.

# SOURCE_DATE_EPOCH is set to 1691114774 (i.e., 20230804T020614Z, timestamp of /etc/apt/sources.list)
FROM ubuntu:jammy-20230804
ENV DEBIAN_FRONTEND=noninteractive
RUN \
  --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  --mount=type=bind,source=./repro-sources-list.sh,target=/usr/local/bin/repro-sources-list.sh \
  repro-sources-list.sh && \
  apt-get update && \
  apt-get install -y gcc

Examples:

Hints

Environment variables

Variable Description Default value
SOURCE_DATE_EPOCH Timestamp of the snapshot (int64) Timestamp of /etc/apt/sources.list, etc. (See below)
WRITE_SOURCE_DATE_EPOCH Write the SOURCE_DATE_EPOCH value to a file /dev/null
SNAPSHOT_ARCHIVE_BASE Base URL of the snapshot http://snapshot-cloudflare.debian.org/archive/, etc. (See below)
BACKPORTS Enable Debian backports 0
KEEP_CACHE Keep apt cache 1

Distribution-specific default values:

Distribution SOURCE_DATE_EPOCH SNAPSHOT_ARCHIVE_BASE
Debian (<= 11) Timestamp of /etc/apt/sources.list http://snapshot-cloudflare.debian.org/archive/
Debian (>= 12) Timestamp of /etc/apt/sources.list.d/debian.sources http://snapshot-cloudflare.debian.org/archive/
Ubuntu (<= 23.10) Timestamp of /etc/apt/sources.list http://snapshot.ubuntu.com/
Ubuntu (>= 24.04) Timestamp of /etc/apt/sources.list.d/ubuntu.sources http://snapshot.ubuntu.com/
ArchLinux Timestamp of /var/log/pacman.log http://archive.archlinux.org/

Related project

https://github.com/reproducible-containers/repro-pkg-cache contains Dockerfile examples to reproduce package cache with specific versions, by pushing the cache to an image registry.

Project Cache location Best for
https://github.com/reproducible-containers/repro-sources-list.sh Distros' permanent snapshot servers (*1) Debian, Ubuntu, ArchLinux
https://github.com/reproducible-containers/repro-pkg-cache Your own permanent image registry Alpine, Fedora, Rocky, openSUSE, etc.

(*1): The packages can be also ephemerally cached on GitHub Actions to reduce loads on distros' snapshot servers. See the Hints above.

repro-sources-list.sh's People

Contributors

akihirosuda avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

repro-sources-list.sh's Issues

Doc: Provide example of building image

I'm trying to build and I'm not getting reproducible images. This is the buildx command I'm using:

docker buildx build -o type=oci,dest=image.img,rewrite-timestamp=true -f Dockerfile.debian-12 --platform linux/amd64 --no-cache --build-arg SOURCE_DATE_EPOCH=1675298208 .

Could there be an example buildx invocation in the README to help someone use this strategy?

FR: a reliable way to fetch the script

Hi! Thanks for your great work :)

The README mentions

# SOURCE_DATE_EPOCH is set to 1691114774 (i.e., 20230804T020614Z, timestamp of /etc/apt/sources.list)
FROM ubuntu:jammy-20230804
ENV DEBIAN_FRONTEND=noninteractive
RUN \
  --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  --mount=type=bind,source=./repro-sources-list.sh,target=/usr/local/bin/repro-sources-list.sh \   # <-- THIS LINE HERE
  repro-sources-list.sh && \
  apt-get update && \
  apt-get install -y gcc

What do you think about showing an example Dockerfile that fetches /repro-sources-list.sh?

I'm thinking this repo could be maintaining a docker image that only contains the script in a scratch image. This could make the setup even simpler:

# SOURCE_DATE_EPOCH is set to 1691114774 (i.e., 20230804T020614Z, timestamp of /etc/apt/sources.list)
FROM ubuntu:jammy-20230804
ENV DEBIAN_FRONTEND=noninteractive
RUN \
  --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  --mount=from=reproducible-containers/repro-sources-list.sh:v1@sha256:...,type=bind,source=/repro-sources-list.sh,target=/usr/local/bin/repro-sources-list.sh \   # <-- THIS LINE HERE
  repro-sources-list.sh && \
  apt-get update && \
  apt-get install -y gcc

The scratch image Dockerfile could be:

# Tag as reproducible-containers/repro-sources-list.sh:v1
FROM scratch
COPY ./repro-sources-list.sh /

Thoughts?

Question: Is it possible to use repro-sources-list.sh to set a date different from (more up-to-date than) the base image tag?

As the examples here in the repo suggest, one can/should pin Ubuntu and Debian images using a date tag, e.g.

FROM ubuntu:jammy-20230804
# or
FROM debian:bookworm-20230904-slim

However, those images only get published occasionally, which causes problems the second a vulnerability gets reported in some package and an update to a package version published after the given date 20230904 is needed.

Is it possible to have repo-sources-list.sh set a more recent snapshot date, independently of the Ubuntu/Debian image tag? In other words, the base image would be e.g. debian:bookworm-20230904-slim but the repo sources would then get set to, say, the 2024-04-07 snapshot from snapshot.debian.org. Of course, this would trigger lots of upgrades during apt-get update && apt-get upgrade later but that is fine / is actually the point.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.