88789cc made TransitiveLicensesInfo private, but that is breaking downstream users of gather_licenses_info.

We need to do that in a soft transition and provide an alternative.

We hold an engineering status meeting on the forth Monday of every month at 12pm USA East coast time.
Add to calendar / meeting notes

The License json object is like:

[
  {
    "top_level_target": "//target:target",
    "dependencies": [
      {
        "target_under_license": "@com_github_fmtlib_fmt//:lib",
        "licenses": [
          "@com_github_fmtlib_fmt//:license"
        ]
      }
    ],
    "licenses": [
      {
        "label": "@com_github_fmtlib_fmt//:license",
        "rule": "@com_github_fmtlib_fmt//:license",
        "license_kinds": [
          {
            "target": "@rules_license//licenses/spdx:MIT",
            "name": "MIT",
            "conditions": []
          }
        ],
        "copyright_notice": "",
        "package_name": "fmtlib",
        "package_url": "",
        "package_version": "",
        "license_text": "external/com_github_fmtlib_fmt/LICENSE.rst",
        "used_by": [
          "@com_github_fmtlib_fmt//:lib"
        ]
      }
    ]
  }
]

The licenses argument of _do_licenses at

[
      {
        "label": "@com_github_fmtlib_fmt//:license",
        "rule": "@com_github_fmtlib_fmt//:license",
        "license_kinds": [
          {
            "target": "@rules_license//licenses/spdx:MIT",
            "name": "MIT",
            "conditions": []
          }
        ],
        "copyright_notice": "",
        "package_name": "fmtlib",
        "package_url": "",
        "package_version": "",
        "license_text": "external/com_github_fmtlib_fmt/LICENSE.rst",
        "used_by": [
          "@com_github_fmtlib_fmt//:lib"
        ]
      }
]
  

but the unique_licenses function try to get licenses field from each item in the licenses list, so it will always be empty.

Specifications

• Version: 0.0.4
• Platform: Windows

Is it possible to extend the aspect such that third party license information would be automatically populated?

For example pip_parse should expose the Python license meta information (setup.py license=, pyproject.toml license=](https://python-poetry.org/docs/pyproject/#license), [License ::` classifier).

• Tests should be less brittle than exact file matching.
• It should be easy to rewrite them when vendoring in to your code base.

Towards the second point, we want to be able to minimize the number of places where we have to rewrite @rules_license to the //path/to/my/rules_license if we import the package into SCM. Test data should not be transformed, and the test itself might have a single constant within it (or injected to the test BUILD target) to provide the root of the tree.

Expected Behavior

The json output produced by licenses_info_to_json() doesnt include include long_name of License_kind. However, we have a need for human-readable license names too.

Actual Behavior

The json output only contains only target, name, and condition. However, we have a need for human-readable license names too.

Steps to Reproduce the Problem

1. Use this aspect for a target and inspect the json output

We should provide //licenses/spdx:* for all of the SPDX identifiers.
That will allow package authors to more precisely declare what license the package is under.
In the future we could add tooling to generate SPDX file from a license() rule, but there are few consumers for that now. We can create an issue for that in the future if we want.

Expected Behavior

As the design doc says,

additional_info: a map of strings to strings. This is a catch-all bucket for users to add custom metadata about this license instance. For example, if this is a paid license, it might indicate if it is site-wide or limited to a number of simultaneous instances. Nothing in this document refers to additional_info again.

Actual Behavior

No such attribute on the rule.

Do we user a docs branch or keep in main?

Currently, running add_licenses.py requires an extra step to be run before it is executed:

wget https://github.com/spdx/license-list-data/raw/master/json/licenses.json

Also, add_licenses.py expects the licenses.json file to be in the correct place when it is executed, and it will fail if it is not found.

Instead, the call to download licenses.json from Github should be incorporated into the add_licenses.py script to remove the need for an extra step and room for user error

See PR review comment: #17 (comment)

• Allow a subtree to be used at runtime, distinct from documents

Today, developers must take care to include sources in the filegroup(name = "standard_package") targets which are sprinkled around the repository.
In the best case, omitting one will be caught by automated testing in the repo. However,

• it's a lot of effort to add sufficient test coverage for this
• we still won't catch everything.

I also claim there is no benefit to having Bazel build the release artifact. git archive is completely sufficient for this task (including pruning directories that are unnecessary in the release artifact and which make it large)

Making this change would simplify the repository so that it's more feasible for us to maintain it with a very small effort.

The applicable_licenses field was renamed inside of Bazel which breaks license collection on the latest 8.x releases of Bazel.

https://github.com/bazelbuild/bazel//commit/e54078fbe408a77a55cdaec7f0b8b35b19ae3010

When you depend on a tool it should not be included in the license/metadata gathering.
We have the filtering function, but it is insufficient.

Some ideas:

• filter any dependency in the exec configuration. Crude but mostly right.
• metadata about the attribute usage on attributes. Elegant but hard.

Let's update this repo to match https://github.com/bazel-contrib/rules-template

The release automation is completely performed in a standard, shared GitHub Actions reusable workflow.

As rule maintainers, this reduces the possibility we can make a mistake during a release, and improves security by allowing GitHub Actions to fully verify the release artifacts. In fact, we'll be able to add some SLSA-style attestation to the bazel-contrib/rules-template in a couple weeks.

It also reduces the maintenance burden for the repo to the minimum necessary, which is important for its long term health as individual engineers get pulled into other things.

There are too many things in the rules package.
The expected use case is that everyone is going to by declaring a license rule, but fewer people will be using those to produce license audit or SBOMs. Most of the people who do produce an SBOM are expected to check rules_license into their repo and modify or reuse as appropriate, so having a clear separation between the packages you modify and the ones you leave alone is helpful.

Expected Behavior

It should be possible to create a license that does not set a license_text.

While it's certainly not a great idea to not have a license_text for a license, that's something that should be enforced (at the user's discretion) in compliance rules, rather than in the structure of the rule declarations. There are many situations where there isn't a practical alternative, e.g. downloading a python wheel where you know the SPDX ID of the license but the file isn't included in the zip file.

Actual Behavior

The macro wrapper sets license_text to LICENSE. This may be convenient in some cases, but at the same time that kind of invisible, implicit source file dependency is generally not the way most things work in bazel.

Expected Behavior

After updating //licenses/spdx/BUILD, all the license_kind definitions should be in alphabetical order by name.

Actual Behavior

After each run, new items are added in alphabetically ordered chunks. That is ugly.

Implementation question: Do we sort with LC_ALL=C or use unicode "human readable" ordering?

Expected Behavior

Should traverse the dependencies, and pickup all licenses

Actual Behavior

The dict is not which are not list and is converted to a list of a dict so the labels are not traversed. Instead the keys from the dict should be traversed.

Steps to Reproduce the Problem

1. Define a rule which takes a attr.label_keyed_string_dict
2. Add a label as an input to that rule which has a license
3. Check that gather licenses using the rule defined above

Specifications

diff --git a/C:/dev/rules_license-0.0.4/rules/licenses_core.bzl b/rules_license/rules/licenses_core.bzl
index 42702bdb7d..fae491539e 100644
--- a/C:/dev/rules_license-0.0.4/rules/licenses_core.bzl
+++ b/rules_license/rules/licenses_core.bzl
@@ -74,7 +74,9 @@ def _get_transitive_licenses(ctx, trans_licenses, trans_deps, traces, provider,
         a = getattr(ctx.rule.attr, name)
 
         # Make anything singleton into a list for convenience.
-        if type(a) != type([]):
+        if type(a) == type({}):
+            a = a.keys()
+        elif type(a) != type([]):
             a = [a]
         for dep in a:
             # Ignore anything that isn't a target

• Version: 0.0.4
• Platform: all

Set up CI on buildkite.

It should be small and unambiguous in favor of comprehensive.

• notice
• requires_relinkability
• unencumbered

Maybe more.

The current meeting is last Monday of the month at 11am Easter time. That is inconvenient for people in Europe and for those in California. Let's see if we can come up with a better day, time, or moving target.

The current one is stale.

While trying out rules_license for a ruleset I maintain, I noticed three problems that made it difficult for me to use:

1. The well-known license_kinds are not publicly visible (fixed by #7).
2. It is not possible to set the visibility of license targets (fixed by #8).
3. There is no release, so the repository can't be safely loaded via http_archive. This turned out to be an "urban legend", see bazel-contrib/SIG-rules-authors#11 (comment).

Is rules_license considered to be in a stable enough state that rulesets could already use it to add applicable_licenses with well-known license_kinds to their public targets? If so, it would be great if these three problems could be resolved.

Expected Behavior

Adding a license file should be optional.

Actual Behavior

When adding attributes for a package under the unlicense license it is missing in the package since it's not a condition of the license to keep the license file when distributing. We can not add just the license-file locally in our repo since glob cannot traverse different namespaces, from @package -> @our_repo//workspace/package:LICENSE.

Currently, the add_licenses.py tool must be run with:

LC_ALL="en_US.UTF-8" admin/refresh_spdx/add_licenses.py

Instead, declare a py_binary target in admin/refresh_spdx/BUILD so the add_licenses.py tool can be run with:

bazel run //admin/refresh_spdx:add_licenses

See PR review comment: #17 (comment)

Expected Behavior

LicensesInfo / LicenseInfo will be collected by reviewing all attributes of objects passed in to the gather_licenses_info aspect.

Actual Behavior

The list of attributes that are inspected for LicenseInfo / LicensesInfo does not gather licenses for all third party libraries included in an android app binary.

Steps to Reproduce the Problem

1. Use gather_licenses_info aspect to collect license information for your deps

Note: the same way as shown in this docstring

Specifications

• We found including exports allows us to find significantly more licenses when building an android binary.

• Version: 0.0.3

• Platform: Android

Some packages include an SBOM produced by publisher.
We should be able to represent that in package_info

Typically, every BUILD file in a bazel module is under the license defined at //:license. Since Bazel does not have inheritance from enclosing packages, we must usually add:

package(default_package_metadata=["//:license", "//:package_info"])

to every BUILD file. We should provide tools to help user maintain that invariant.

Initial thoughts:

• Must be user extensible so organizations can add their own policies.
• Standalone tool:
  • pro: can find all files, even if they are not in a package. That is a big win if you want to inject license scanning at this point.
  • con: can't be a bazel test.
• Bazel rule
  • pro: Can be a test, so it is trivial to add to CI.
  • con: Misses files which are not mentioned from a BUILD file.

@jin

Trying to apply the license collection mechanism outlined in the examples/manifest directory onto another Bazel based project fails at load("@rules_license//rules:compliance.bzl", "manifest") in the WORKSPACE file with error Error: file '@rules_license//rules:compliance.bzl' does not contain symbol 'manifest'.

Issuing a load("@rules_license//rules:license_kind.bzl", "license_kind") in the same WORKSPACE file works fine, though, so the rules_license http_archive was found and properly loaded.

That same load(...) works when building inside the examples/manifest directory, which makes me suspect the visibility is somewhat wrong or is the examples/manifest something that is meant to be looked at when trying out license collection? Is the proper way to use rules_license in ones own project described somewhere in a way that a novice Bazel user could evaluate its functionality?

exports_files declarations should be traversed by the gathering aspect, although they are commonly used to import various licensed resources into build graphs.

Unfortunately, it looks like aspects don't traverse exports_files targets at all, as these are not standard bazel rules. Furthermore, exports_files also does not supports the applicable_licenses attribute.

Hello,

I have to list all the external dependencies of a bazel built project (envoy), with their name, version and license(s).

It seems rules_license would help me in this task, but I don't understand how to use it (I am new to bazel), do you have an example ?

Regarding my envoy specific use case, it seems all the required information is available to bazel: envoy/bazel/repository_locations.bzl

I have a bazel build command used to produce an envoy static binary with many 'filters' and 'options':

bazel \
  build \
  --define google_grpc=disabled  \
  --@envoy//bazel:http3=False     \
  --@envoy//source/extensions/wasm_runtime/v8:enabled=false \
  [...]
  @envoy//source/exe:envoy-static

https://github.com/bazelbuild/bazel//commit/e54078fbe408a77a55cdaec7f0b8b35b19ae3010

I couldn't find any issue tracking this but please enlighten me if that is the case. Has there been any thoughts on creating a tool for gathering the license info and creating a BOM in the SPDX format? I am no expert, but to me it looks like the .json produced by the license_used() rule is not in the SPDX format.

Would that be in the scope of this library or something that users should be responsible for?

Expected Behavior

Only the path to the license file is available in the JSON.

Actual Behavior

Have the actual content of the license file available in the JSON.

Steps to Reproduce the Problem

1. Run aspect to generate the licenses JSON output

Specifications

• Version: latest
• Platform: Linux

Expected Behavior

Rule Usage example:

load("@rules_license//rules:compliance.bzl", "check_license")

check_license(
    name = "third_party_licenses",
    license_texts = "third_party_licenses.txt",
    report = "third_party_reports.txt",
    deps = _ROOT_TARGETS,
)

Actual Behavior

ERROR: C:/users/dhmem/***/BUILD:116:14: in _check_license rule //third_party_licenses:                                                                    
Traceback (most recent call last):
        File "C:/users/dhmem/_bazel_dhmem/36t2vbit/external/rules_license/rules/compliance.bzl", line 33, column 24, in _check_license_impl
                write_licenses_info(ctx, ctx.attr.deps, licenses_file)
        File "C:/users/dhmem/_bazel_dhmem/36t2vbit/external/rules_license/rules/gather_licenses_info.bzl", line 144, column 50, in write_licenses_info
                licenses.extend(licenses_info_to_json(dep[TransitiveLicensesInfo]))
        File "C:/users/dhmem/_bazel_dhmem/36t2vbit/external/rules_license/rules/gather_licenses_info.bzl", line 245, column 58, in licenses_info_to_json
                top_level_target = _strip_null_repo(licenses_info.target_under_license),
Error: 'TransitiveLicensesInfo' value has no field or method 'target_under_license'
Available attributes: deps, licenses, traces
ERROR: C:/users/dhmem/***/BUILD:116:14: Analysis of target '//third_party_licenses' failed                                                                
ERROR: Analysis of target '//third_party_licenses' failed; build aborted:                                                                                                           
INFO: Elapsed time: 0.696s                                                                                                                                                               
INFO: 0 processes.                                                                                                                                                                       
FAILED: Build did NOT complete successfully (1 packages loaded, 0 targets configured)

Caused by an empty TransitiveLicensesInfo:

DEBUG: C:/users/dhmem/_bazel_dhmem/36t2vbit/external/rules_license/rules/gather_licenses_info.bzl:243:10: licenses_info.target_under_license:  struct(deps = depset([]), licenses = depset([]), traces = [])

And the empty TransitiveLicensesInfo is from

Specifications

• Version: 0.0.4
• Platform: Windows

Expected Behavior

Custom starlark rules should be able to return a LicensesInfo provider in the event that they need to customize how licenses propagate. For example, some rules may need to propagate licenses through additional attributes which are unique to that rule. Others may for example want to filter the transitive license set based on the conditions on the license kinds.

Actual Behavior

gather_licenses_info aspect adds a LicensesInfo provider unconditionally, which results in an error if the target already has one. It should instead check whether the target already has a LicensesInfo provider and return an empty provider list if it does.

The JSON output produced by gather_licenses_info_and_write includes unnexpected dependencies and licenses.

Actual Behavior

There should be a way to filter these targets out

Steps to Reproduce the Problem

1. Apply gather_licenses_info_and_write aspect to a pkg_rpm() target like:

pkg_rpm(
    name = "xyz-rpm",
    srcs = [...]
    ...
)

, which transitively consumes some deps like postgresql, javax-mail etc.
2. The JSON output produces by the aspect reports @rules_pkg//pkg:make_rpm as a dependency target in addition to correctly identifying postgresql, javax-mail etc. There should be a way to filter out these targets as dependencies and their associated licenses.

Sample output can be found here:
https://gist.github.com/rtabassum/b9de682c2939f7e0b882d86f8686ed12#file-xyz-licenses-json

Specifications

• Version:
• Platform:

Plan here: Adding OSS license declarations to Bazel

I just wanted to move our requirements from a comment on a closed PR (#6 (comment)) to the bugtracker:

• it didn't collect through output file dependencies (fixed now)
• we have a custom license type that also has attached file references to license and notice files as well as URLs pointing at the original source (this information is required for some cloud marketplace deployments)
• the list of attributes it traverses is too narrow; it's actually hard to come up with a complete list, but here's what we're using for a Java application: "deps", "exports", "jars", "resources", "runtime_deps", "srcs"; for other languages, we may need to collect from even more attributes (e.g., hdrs for C++)
• it doesn't enforce our repo-specific annotation rules: we're enforcing license annotations for all rules under our third_party directory and a subset of external workspaces; some of that is specific to our use case

When rules_jvm_external creates BUILD files, it should add package_info() targets.

This will be fixed in the 0.0.7 release. compliance.bzl is mostly deprecated.

Need /licenses/generic:none, notice, unencumbered,
and maybe a few others.

Expected Behavior

When running bazel build //examples/sboms:write_sbom_sbom, the resulting JSON should look as follows:

[
  {
    "top_level_target": "//tools:write_sbom",
    "dependencies": [
      {
        "target_under_license": "//tools:write_sbom",
        "licenses": [
          "//:license"
        ]
      }
    ],
    "licenses": [
      {
        "label": "//:license",
        "bazel_package": "//",
        "license_kinds": [
          {
            "target": "@//licenses/spdx:Apache-2.0",
            "name": "Apache-2.0",
            "conditions": []
          }
        ],
        "copyright_notice": "",
        "package_name": "rules_license",
        "package_url": "",
        "package_version": "0.0.7",
        "license_text": "LICENSE",
        "used_by": [
          "//tools:write_sbom"
        ]
      }
    ],
    "packages": [
          {
            "target": "//:package_info",
            "bazel_package": "//",
            "package_name": "rules_license",
            "package_url": "",
            "package_version": "0.0.7"
          }
    ]
  }
]

Actual Behavior

When running bazel build //examples/sboms:write_sbom_sbom, the resulting JSON actually looks as follows:

[
  {
    "top_level_target": "//tools:write_sbom",
    "dependencies": [
      {
        "target_under_license": "//tools:write_sbom",
        "licenses": [
          "//:license"
        ]
      }
    ],
    "licenses": [
      {
        "label": "//:license",
        "bazel_package": "//",
        "license_kinds": [
          {
            "target": "@//licenses/spdx:Apache-2.0",
            "name": "Apache-2.0",
            "conditions": []
          }
        ],
        "copyright_notice": "",
        "package_name": "",
        "package_url": "",
        "package_version": "",
        "license_text": "LICENSE",
        "used_by": [
          "//tools:write_sbom"
        ]
      }
    ],
    "packages": [
          {
            "target": "//:package_info",
            "bazel_package": "//",
            "package_name": "rules_license",
            "package_url": "",
            "package_version": "0.0.7"
          },
          {
            "target": "//:package_info",
            "bazel_package": "//",
            "package_name": "rules_license",
            "package_url": "",
            "package_version": "0.0.7"
          }
    ]
  }
]

Specifically, the same package is included twice and the license doesn't contain package_name and package_version fields.

Steps to Reproduce the Problem

1. Run bazel build //examples/sboms:write_sbom_sbom
2. Inspect bazel-bin/examples/sboms/_write_sbom_sbom_licenses_info.json

Specifications

• Version:
• Platform:

• pull information from (design)[https://docs.google.com/document/d/1uwBuhAoBNrw8tmFs-NxlssI6VRolidGYdYqagLqHWt8/edit#]
• and from licenses/spdx/BUILD
• Use case for just labeling your work
  • depend on the release and add license() rules
• Use case for building license inclusion reporting and testing
  • Some reporting for free
  • Recommend that you vendor package in for meaningful compliance

Expected Behavior

gather_licenses_info.bzl should also gather license from external packages (fetched thru WORKSPACE or Bzlmod).

Actual Behavior

gather_licenses_info.bzl only gathers license from in-tree project sources and vendored dependencies.

Steps to Reproduce the Problem

git clone https://github.com/google/xls
cd xls
bazel build //xls/dslx:interpreter_main --aspects=@rules_license//rules:gather_licenses_info.bzl%gather_licenses_info_and_write --output_groups=licenses

xls 🍡  cat bazel-bin/xls/dslx/interpreter_main_licenses_info.json | grep @
            "target": "@rules_license//licenses/spdx:Apache-2.0",

xls 🍙  bazel query --color=no --noshow_progress --noshow_loading_progress --notool_deps --noimplicit_deps 'kind(cc_library, filter("^@", deps(//xls/dslx:interpreter_main)))' --output package | grep @ | wc -l
39

Specifications

• Version: bazel 6.4.0
• Platform: Linux penguin 6.1.64-09049-g010fe86d9eae #1 SMP PREEMPT_DYNAMIC Thu, 1 Feb 2024 01:25:43 +0000 x86_64 GNU/Linuxw

We should be able to generate docs from the .bzl files.

That require using bz_library rules, but that would take a dependency on bazel-skylib, which impedes reuse.
Blocked on bazelbuild/bazel-skylib#127 (providing we resolve that by making bzl_library part of @bazel_tools)

Currently, it is a manual process to check that the list of SPDX licenses represented by licenses/spdx/BUILD are up-to-date.

Instead, add a CI test that checks what we have against the authoritative source and fails if we are more than some threshold behind.

See PR review comment: #17 (comment)

Expected Behavior

All examples work.

Actual Behavior

$ bazel test //examples/manifest:main_test
ERROR: /workspaces/rules_license/examples/manifest/BUILD:25:15: in cmd attribute of genrule rule //examples/manifest:gen_main_manifest: label '//examples/manifest:main_manifest' in $(locations) expression expands to no files. Since this rule was created by the macro 'android_binary', the error might have been caused by the macro implementation
ERROR: /workspaces/rules_license/examples/manifest/BUILD:25:15: Analysis of target '//examples/manifest:gen_main_manifest' failed
ERROR: Analysis of target '//examples/manifest:main_test' failed; build aborted: 
INFO: Elapsed time: 0.154s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 6 targets configured)
ERROR: Couldn't start the build. Unable to run tests

Steps to Reproduce the Problem

1. bazel test //examples/manifest:main_test

Specifications

• Version:
• Platform:

https://github.com/bazelbuild/rules_license/blob/main/rules/license.bzl#L112 prevents specifying license texts that are labels.

My project (Fuchsia) needs label support for:

1. To take advantage of "select(...)", which is needed when licenses are different under certain build conditions.
2. Some of our licenses reside outside of the Bazel package folder and can't be glob. That is because Fuchsia assembles system out of prebuilts from other repositories and the tools used to fetch these prebuilts + licenses (e.g. Jiri or git repo) typically do that into a separate place from the Bazel rules that wrap them.

To solve, once can implement the above check at the rule itself, instead of in the macro. e.g.

 if not ctx.file.license_text.is_source:
        fail("License file ", ctx.file.license_text.path, "must be a source file, not generated")