bazel-contrib / rules-template Goto Github PK

The mylang_register_toolchains macro invokes toolchains_repo which generates a toolchain target for each platform as well as a resolved_toolchains target that acts as an alias for the currently resolved toolchain. This is intended as a workaround for bazelbuild/bazel#14009 to expose make variables to rules that want to directly depend on the "mylang" toolchain, e.g. genrule.

Because the resolved_toolchains target is generated as part of toolchains_repo users have to know and use the name given to that external workspace. E.g. in the example setup a user would have to use the label @mylang1_14_toolchains//:resolved_toolchain. This is problematic:

• If the target is needed by a Bazel workspace other than the main workspace, that workspace has to either anticipate the name that the main workspace will assign to that toolchain repository, or import its own "mylang" toolchain which may conflict with that needed by the main workspace or another workspace.
• If a project needs a custom toolchain and doesn't invoke mylang_register_toolchains or toolchains_repo at all, then there will be no resolved_toolchains target.

There is no reason why resolved_toolchains has to part of or linked to toolchains_repo or mylang_register_toolchain. I propose that resolved_toolchains should be defined in com_myorg_rules_mylang directly, instead.

See https://copier.readthedocs.io/en/stable/

This will allow to do the:

• search for "com_myorg_rules_mylang" and replace with the name you'll use for your workspace
• search for "myorg" and replace with GitHub org
• search for "mylang", "Mylang", "MYLANG" and replace with the language/tool your rules are for
• rename directory "mylang" similarly

steps for the automatically user after asking for those details.

Also, when this template involves, users can use Copier to pull in those changes to their instantiated projects. Ref: https://copier.readthedocs.io/en/stable/updating/

With the upcoming release of 6.0.0 would it make sense to set the default implementation for new rule sets to use bzlmod by default?

Could we at least consider showing some side by side examples in the same style shown here? https://github.com/bazelbuild/examples/tree/main/bzlmod/04-local_config_and_register_toolchains

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency bazel to v7.1.2
• chore(deps): update dependency bazel_skylib to v1.7.1

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

I recently observe in https://github.com/aherrmann/rules_zig/ that the toolchain is managed via a rule called zig_repository, which follow the template convention of mylang_repository.

However, what I am used to in Go is go_repository is used to manage 3rd party dependency import. Toolchain in Go is managed via go_{download,local,host}_sdk and go_register_toolchains rules instead.

So I would suggest renaming mylang_repository to mylang_sdk and mylang_register_toolchains while reserving mylang_repository for respective language's 3rd party import. WDYT?

Note: fix for this already exists in #35 , just providing additional context.

After forking this, I noticed that the cache actions were failing to persist with the following error

Warning: Path Validation Error: Path(s) specified in the action for caching do(es) not exist, hence no cache is being saved.

Which was surprising because these paths do, in fact, exist! After reading the yaml docs on how the vertical bar operator works, my understanding is that

path: "~/.cache/bazel"

results in

~/.cache/bazel

in github's action processor, whereas

path: |
  "~/.cache/bazel"
  "~/.cache/bazel-repo"

results in

"~/.cache/bazel"
"~/.cache/bazel-repo"

because it doesn't unquote, and the processor isn't handling that at all. Switching to

path: |
  ~/.cache/bazel
  ~/.cache/bazel-repo

fixes the caching.

The template defined in this repository and the corresponding GitHub actions configuration configure a --repository_cache to be cached with the GitHub actions cache. However, in my experience, fetching artifacts from the GitHub actions cache is often slower than fetching the artifacts directly from the internet in a GitHub actions run.

I've set up a simple example to illustrate this: In that case we use Bazel to download binaries of varying sizes and either use Bazel's repository cache feature to cache on GitHub actions cache or not. In the case of a download artifact of about 200MiB in size the full build step (including the download of Bazel itself) takes about 9s. However, with the GitHub actions cache enabled the download of the cache alone takes 1m6s. According to the cache action's outputs the download itself is fast at somewhere around 130MB/s, but the overall step duration brings this down to about 3MB/s. (Perhaps the decompression takes so long.)

On smaller sized artifacts the difference vanishes, the GitHub actions run takes about the same time whether the repository cache is used or not. So, in such cases the repository cache doesn't add much value, but only drives up the storage used.

This begs the question, should we recommend a repository cache in the rules template, or is it better to remove it from the example CI configuration?

(Relevant changes d017a1d and aherrmann/.github@07b6368)

In bazelbuild/rules_pkg#716 (comment) there's a suggestion that rules need complexity in their release process in order to make them secure.

I believe we have already followed excellent security practices in the way releases are created here. By only allowing a developer with write access to push a tag, there's no way to create a release with contents that aren't reproducible from the sources.

I'd be very interested to know of any vulnerabilities with our approach, as there are now dozens of rulesets based on this template.

Many languages require building and using supplementary tools, like @io_bazel_rules_go's use of a builder tool. Is there guidance on the "right" way to author tools which are used by this rules-template, and hook them up to the ruleset? What about if mylang is not an appropriate language in which to author these tools?

In the past, I've written these tools as:

1. sh_binary()s, which doesn't work out of the box on Windows particularly well, and often depends on other system tools
2. py_binary()s, which don't work well in Bazel remote build scenarios, particularly when using python3 and third-party pip libraries
3. go_binary()s, which end up polluting my rulesets with external references to @io_bazel_rules_go

There seems to be an issue with the buildifier check on Windows CI.

The error is the following

ERROR: D:/a/rules-template/rules-template/.github/workflows/BUILD.bazel:3:11: While resolving toolchains for target //.github/workflows:buildifier.check: No matching toolchains found for types @buildifier_prebuilt~6.1.0//buildifier:toolchain.

The first occurrence seems to have been here

I experienced some problem as I released a project that is based on rules-template.

1. v0.0.0 - ❌ failed.
2. I fixed it by setting the permissions key in the workflow as described in this comment
3. v0.0.1 - ✔️ succeeded
4. reverted the commit and, instead, set the workflow permissions in the settings
   
5. v0.0.2 - ✔️ succeeded

It seems setting the permissions in the settings or in the workflow file, respectively, fixes the issue.

Did anybody else had this problem? Is this worth a note in the README.md?

I'm thankful for any feedback 🤘