Dockerfile statements:

EXPOSE 80/tcp
LABEL version="1.0"
VOLUME /data
STOPSIGNAL SIGKILL

Should have equivalent cargo-wharf metadata:

[package.metadata.wharf.output]
expose = ["80/tcp"]
labels = { version = "1.0" }
volumes = ["/data"]
stop_signal = "SIGKILL"

I get rpc error: code = Unknown desc = Unable to build the crate => caused by: Unable to solve the graph => caused by: grpc-status: Unknown, grpc-message: "unknown API capability file.base"
After all the dependencies have been downloaded and => Evaluating the build plan 19.3s has been emitted.

# syntax = denzp/cargo-wharf-frontend:v0.1.0-alpha.1

[package]
name = "xxx"
version = "0.1.0"
authors = ["xxx <xxx>"]
edition = "2018"

[dependencies]
actix-cors = "0.2.0"
actix-web = {version = "2.0"} #, features = ["openssl"]}
actix-web-httpauth = "0.4.1"
actix-http = "1.0.1"
actix-rt = "1.0"
actix-threadpool = "0.3.1"
strum_macros = "0.18.0"
chrono = "0.4"
dotenv = "0.15.0"
failure = "0.1.7"
mongodb = "0.9.2"
bson = "0.14.1"
serde = "1.0.105"
serde_derive = "1"
serde_json = "1"
futures = "0.3.4"
validator = "0.10.0"
validator_derive = "0.10.0"
env_logger = "0.7.1"
derive_more = "0.99.3"
log = "0.4.8"
rust-argon2 = "0.8.1"
rand = "0.7.3"
jsonwebtoken = "7.1.0"
openidconnect = {git = "https://github.com/ruseinov/openidconnect-rs.git", features = ["futures-03", "reqwest-010"], default-features = false }
url = "2.1.1"
openssl = "0.10.28"
error-chain = "0.12.2"
reqwest = { version = "0.10", features = ["json"] }

[package.metadata.wharf.builder]
image = "rust:1.42"

[package.metadata.wharf.output]
image = "rust:1.42.0-slim-buster"
workdir = "/"
entrypoint = ["/bin/api"]
expose = ["8088"]
stop-signal = "SIGINT"

[[package.metadata.wharf.binary]]
name = "api"
destination = "/bin/api"

This builds fine with a normal Dockerfile and the same builder image.

There are plenty of missing features that might block migration from Dockerfile-approach.
Some of the features are trivial to implement, others might require extra work and refactoring.

Builder:

• Custom setup commands (RUN directives before cargo build) #15
• Static assets from the build context (COPY directives) #18
• .dockerignore #16

Output:

• Labels #17
• Exposed ports #17
• Volumes #17
• Stop signal #17
• Static assets from the build context (COPY directives) #18
• Custom pre-setup commands (RUN directives before copying build outputs) #15
• Custom post-setup commands (RUN directives after copying build outputs) #15

Efficiently cache crate dependencies.

I'd like to discuss the future of cargo-wharf and to this extent some ideas I'd like to collaborate on.

Cache integration and the case for a community-backed global cache

Recent versions of docker build support --output=PATH which copies files out of an image. This allows for writing the compilation results of each dependency to the filesystem of the local machine or of a CI cache.
cargo has a way of specifying where to look for build artifacts other than the sometimes-empty ./target/ dir: CARGO_TARGET_DIR.

More on CARGO_TARGET_DIR

Per https://stackoverflow.com/a/37472558/1418165 it turns out that a shared CARGO_TARGET_DIR (or CARGO_BUILD_TARGET_DIR)

• is thread safe, since rust-lang/cargo#354 is closed
• names that folder "target" per rust-lang/cargo#1657 (comment)
• however cargo provides no hermeticity garanties WRT
  • feature flags
  • compiler version
  • platform triplet

These would have to be part of the hashed name of each dependency being built (the dependency path or the docker tag).
To solve hermeticity issues, see

• https://github.com/google/cargo-raze
• https://github.com/bazelbuild/rules_rust

cross

cross already does a good job of building Rust projects (on various platform triplets) using docker (docker run) and QEMU.
This work should be adapted (in a way that can be most easily maintainable) to use BuildKit, its QEMU integration, its rootless capabilities and its ability to run the compute graph with maximum parallelization.

Conclusion

So if cargo-wharf where to create hermetic BuildKit targets for each dependency, leveraging the work on cross, I think there'd be a seamless way to integrate both local and global caches for dependencies. This global cache (basically a Docker Registry) could then be paid for by the community and profit the community.

To get there I see these development steps:

• get the list of dependencies from cargo-wharf, hashed and hermetic
• "generate" a Dockerfile with these, based on cross's.
  • Each dep is a stage in this file. Stage name = hashed recipe
  • When linking, dependencies are bind --mount=from=HASHEDDEP,source=...,target=... as read-only
• docker build this Dockerfile as the cargo build equivalent. Same for cargo test.
• In a local cache setting
  • each hashed dependency build results would live in a centralized folder, ready for reuse by another project. Thus lowering initial build times.
• If using the global cache
  • each hashed dependency build results would live as a single-layer docker image, holding files, in the local docker registry as well as the global networked one.
  • New builds should be received by the global registry and checked for hermeticity before adding them to its cache.

Note that that global docker registry

• can easily be switched to a private instance
• could be used to directly build dependencies and/or profit from cache locality by setting it as the docker host in about this way: DOCKER_HOST=ssh://lotsa.oompf.machine.com cargo build, only the final build results would then be transferred over the network.

Ideas, thoughts, notes, criticism please shoot.

Build fails with the error message:

=> Evaluating the build plan 68.1s
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = Unable to evaluate the Cargo build plan => caused by: Unable to parse Cargo build plan => caused by: unknown variant `staticlib`, expected one of `lib`, `bin`, `test`, `custom-build`, `proc-macro`, `example` at line 1 column 149974

The file should be taken into account and have a consistent behaviour with Dockerfile frontend.

Currently, the container-tools image is using a naive build approach: Dockerfile and cargo build inside. It would probably be a nice real world use case if cargo-wharf handle the image creation.

Expected Behavior

When compiling actix-web using cargo wharf with docker build kit, the crate should compile

Compiled fine with a normal docker file, doesn't with cargo wharf.

Current Behavior

Errors with the following message

------
 > Compiling actix-web-codegen:
#242 2.736 error[E0432]: unresolved import `proc_macro`
#242 2.736   --> /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/actix-web-codegen-0.4.0/src/lib.rs:63:5
#242 2.736    |
#242 2.736 63 | use proc_macro::TokenStream;
#242 2.736    |     ^^^^^^^^^^ help: a similar path exists: `syn::proc_macro`
#242 2.754 
#242 3.138 error: aborting due to previous error
#242 3.138 
#242 3.138 For more information about this error, try `rustc --explain E0432`.
------
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = Unable to build the crate => caused by: Unable to solve the graph => caused by: grpc-status: Unknown, grpc-message: "failed to build LLB: executor failed running [rustc --edition=2018 --crate-name actix_web_codegen /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/actix-web-codegen-0.4.0/src/lib.rs --color never --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C opt-level=3 -C metadata=a3bb0cb5e1cde293 -C extra-filename=-a3bb0cb5e1cde293 --out-dir /target/release/deps -L dependency=/target/release/deps --extern proc_macro2=/target/release/deps/libproc_macro2-d94ec969ac1bef7e.rlib --extern quote=/target/release/deps/libquote-0451cbe6c2d2830e.rlib --extern syn=/target/release/deps/libsyn-2ff5fc6c29cb0289.rlib --cap-lints allow]: runc did not terminate sucessfully"

Possible Solution

no idea but it suggests changing the import

Steps to Reproduce (for bugs)

1. Setup Docker build kit and try and compile actix-we with cargo wharf

ex: https://github.com/Daggy1234/dagpi-auth

Context

Its hard to compile actix web with build kit

Your Environment

• Rust Version (I.e, output of rustc -V): Docker(1.48.0)
• Actix Web Version: 3.3.0

Dear maintainer,

I just discover your project and it's absolutely what I was looking for, thanks a lot for your amazing work ! I'd like to report an issue when specifying a builder image.

01. Problem description

I tried to use image = "rust:1.40" in [workspace.metadata.wharf.builder] but it failed to resolve with the following error:

DOCKER_BUILDKIT=1 docker build . -f Cargo.toml
[+] Building 3.6s (11/12)
 => [internal] load build definition from Cargo.toml                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 458B                                                                                                                                                                                                     0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                        0.0s
 => => transferring context: 34B                                                                                                                                                                                                         0.0s
 => resolve image config for docker.io/denzp/cargo-wharf-frontend:v0.1.0-alpha.0                                                                                                                                                         1.3s
 => CACHED docker-image://docker.io/denzp/cargo-wharf-frontend:v0.1.0-alpha.0@sha256:a767ca2b17f8133dfe79c7ac455db25076dc75a29866a37b255d7e10d61dde9d                                                                                    0.0s
 => Using build context                                                                                                                                                                                                                  0.2s
 => => transferring dockerfile: 69.23kB                                                                                                                                                                                                  0.2s
 => CACHED docker-image://docker.io/denzp/cargo-container-tools:v0.2.0-alpha.0                                                                                                                                                           0.0s
 => => resolve docker.io/denzp/cargo-container-tools:v0.2.0-alpha.0                                                                                                                                                                      0.6s
 => Collecting configuration metadata                                                                                                                                                                                                    0.8s
 => Resolving builder image                                                                                                                                                                                                              0.0s
 => Resolving output image                                                                                                                                                                                                               0.0s
 => CANCELED Using build context                                                                                                                                                                                                         0.0s
 => => transferring context:                                                                                                                                                                                                             0.0s
 => ERROR docker-image://docker.io/library/rust:1.40@                                                                                                                                                                                    0.0s
 => => resolve docker.io/library/rust:1.40@                                                                                                                                                                                              0.0s
------
 > docker-image://docker.io/library/rust:1.40@:
------
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = Unable to evaluate the Cargo build plan => caused by: Unable to evaluate the build plan => caused by: Unable to solve the graph => caused by: grpc-status: Unknown, grpc-message: "failed to build LLB: failed to load cache key: invalid reference format"

02. Note

It works fine with image = "rust"

03. Versions

Client: Docker Engine - Community
 Version:           19.03.5

Server: Docker Engine - Community
 Engine:
  Version:          19.03.5

We try to build a container image via cargo wharf. Locally everything works fine. However, in github actions we have failures at random crates (everytime a different one). The errors always look like this:

#375 Compiling unic-char-range
#375 ERROR: executor failed running [rustc --edition=2018 --crate-name unic_char_range /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/unic-char-range-0.9.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 --cfg feature="default" -C metadata=3b7b6d2f09242336 -C extra-filename=-3b7b6d2f09242336 --out-dir /target/release/deps -L dependency=/target/release/deps --cap-lints allow]: flightcontrol: exceeded retry timeout

We suspect it might be due to the implicit parallelization. Is it possible to limit it somehow? Or do you have another hint about the error?

Cargo adds metadata hash which also depends on Rust compiler version. Therefore, after each host Rust compiler update, the Docker cache will be invalidated (because rustc arguments will be updated) even if Rust compiler in builder image remains the same.

Even more important, Docker has own way to invalidate cache based on build image hash, so must be okay to override metadata created by Cargo.

The behaviour should be consistent with cargo-publish and --allow-dirty flag.

Regardless of passing --ssh=default or not into the docker build, the option is not being used at the moment.

# #12 Evaluating the build plan
# #12 0.849     Updating git repository `ssh://[email protected]/nix-rust/nix.git`
# #12 1.541 error: failed to load source for a dependency on `nix`
# #12 1.541
# #12 1.541 Caused by:
# #12 1.541   Unable to update ssh://[email protected]/nix-rust/nix.git
# #12 1.541
# #12 1.541 Caused by:
# #12 1.541   failed to clone into: /usr/local/cargo/git/db/nix-cac03638f82838d2
# #12 1.541
# #12 1.541 Caused by:
# #12 1.542   failed to authenticate when downloading repository
# #12 1.542 attempted ssh-agent authentication, but none of the usernames `git` succeeded
# #12 1.542
# #12 1.542 Caused by:
# #12 1.542   error authenticating: no auth sock variable; class=Ssh (23)
# #12 1.544 error: Child process failed
# #12 ERROR: executor failed running [/usr/local/bin/cargo-build-plan --manifest-path /context/Cargo.toml --output /output/build-plan.json --release]: buildkit-runc did not terminate sucessfully

Dockerfile statements:

RUN ["sudo", "chown", "-R", "rust:rust", "/rust-src"]
RUN apt-get install ffmpeg

Should have equivalent cargo-wharf metadata:

[package.metadata.wharf.builder]
setup-commands = [
  { command = ["sudo", "chown", "-R", "rust:rust", "/rust-src"] },
  { shell = "apt-get install ffmpeg" }
]

[package.metadata.wharf.output]
pre-install-commands = [
  { command = ["sudo", "chown", "-R", "rust:rust", "/rust-src"] },
  { shell = "apt-get install ffmpeg" }
]

post-install-commands = [
  { shell = "lld /bin/exec1"}
]

Dockerfile statements:

COPY hom* /mydir/ 
COPY hom?.txt /mydir/

Should have equivalent cargo-wharf metadata:

[package.metadata.wharf.builder]
copy = [
  { src = "hom*", dst = "/mydir/" },
  { src = "hom?.txt", dst = "/mydir/" }
]

[package.metadata.wharf.output]
copy = [
  { src = "hom*", dst = "/mydir/" },
  { src = "hom?.txt", dst = "/mydir/" }
]

Error response from daemon: Dockerfile parse error line 2: unknown instruction: [WORKSPACE]

I have to use the silly workspace hack from cargo, but wharf doesn't seem to support an empty workspace tag.

Currently, the application uses Handlebars templates to render Dockefile, but this approach relies on assumption that Tests and Binaries stages are the last stages in the template. The reason for this, is the fact that we don't know the stages' names, so we can't tell docker to build a particular stage docker build --target STAGE_NAME, instead it builds only last stage and it's dependencies.

As an alternative approach, we can do a syntactic and semantic analysis of the Dockerfile template. We would analyse existing stages, and directives which define where should we place generated stages and outputs.

Is there a way to allow this to output multiple images?

I'm using a workspace, but rather than output 2 binaries in the same image, I'd love to have this output 2 images each with one of the binaries.

If there is somewhere you can point me to implement this please do, I'm happy to spend the time!

Build script sequence consists of several steps:

1. compile build script binary
2. run build script binary
3. compile the original crate

Without additional efforts, all those steps are being executed in different stages, which means they can't affect each other. But the idea of build scripts is that they could alter the original crate building.
We need to ensure steps [2] and [3] are being run in a single stage and we take in account build script output.

Cache export and import from remote registry with is currently useless:

docker buildx build --cache-from ... --cache-to ...

Normally the approach should speed up CI times dramatically.

This is needed for integration tests to use the correct local cargo-container-tools image.

docker build --build-arg features=feature1,feature2

Blocks #8

Hi @denzp! First of all, many many thanks for this kick-ass project. I use it for all of my dockerized rust projects and it is a truly great project. I'm amazed it doesn't have thousands of stars, to be honest.

I'm very much interested in support for Dockerfile's COPY (#18), and I have started playing around with implementing it in cargo-wharf (you can look at my [quite little] progress at https://github.com/denzp/cargo-wharf/compare/master...edrevo:add-copy-support?expand=1). One thing I'm missing is a bit of developer documentation on the repo. Specifically, how do I run the integration tests? It might be useful to document that, or even better, just add a bash script to run them, so it is straighforward.