bazelbuild / rules_license Goto Github PK
View Code? Open in Web Editor NEWLicense: Apache License 2.0
License: Apache License 2.0
Is it possible to extend the aspect such that third party license information would be automatically populated?
For example pip_parse
should expose the Python license meta information (setup.py license=
, pyproject.toml license=](https://python-poetry.org/docs/pyproject/#license), [
License ::` classifier).
license()
rulesIt should be small and unambiguous in favor of comprehensive.
Maybe more.
The current meeting is last Monday of the month at 11am Easter time. That is inconvenient for people in Europe and for those in California. Let's see if we can come up with a better day, time, or moving target.
We hold an engineering status meeting on the forth Monday of every month at 12pm USA East coast time.
Add to calendar / meeting notes
Hello,
I have to list all the external dependencies of a bazel built project (envoy), with their name, version and license(s).
It seems rules_license
would help me in this task, but I don't understand how to use it (I am new to bazel), do you have an example ?
Regarding my envoy
specific use case, it seems all the required information is available to bazel: envoy/bazel/repository_locations.bzl
I have a bazel
build command used to produce an envoy static binary with many 'filters' and 'options':
bazel \
build \
--define google_grpc=disabled \
--@envoy//bazel:http3=False \
--@envoy//source/extensions/wasm_runtime/v8:enabled=false \
[...]
@envoy//source/exe:envoy-static
Should traverse the dependencies, and pickup all licenses
The dict is not which are not list and is converted to a list of a dict so the labels are not traversed. Instead the keys from the dict should be traversed.
attr.label_keyed_string_dict
diff --git a/C:/dev/rules_license-0.0.4/rules/licenses_core.bzl b/rules_license/rules/licenses_core.bzl
index 42702bdb7d..fae491539e 100644
--- a/C:/dev/rules_license-0.0.4/rules/licenses_core.bzl
+++ b/rules_license/rules/licenses_core.bzl
@@ -74,7 +74,9 @@ def _get_transitive_licenses(ctx, trans_licenses, trans_deps, traces, provider,
a = getattr(ctx.rule.attr, name)
# Make anything singleton into a list for convenience.
- if type(a) != type([]):
+ if type(a) == type({}):
+ a = a.keys()
+ elif type(a) != type([]):
a = [a]
for dep in a:
# Ignore anything that isn't a target
gather_licenses_info.bzl
should also gather license from external packages (fetched thru WORKSPACE
or Bzlmod
).
gather_licenses_info.bzl
only gathers license from in-tree project sources and vendored dependencies.
git clone https://github.com/google/xls
cd xls
bazel build //xls/dslx:interpreter_main --aspects=@rules_license//rules:gather_licenses_info.bzl%gather_licenses_info_and_write --output_groups=licenses
xls ๐ก cat bazel-bin/xls/dslx/interpreter_main_licenses_info.json | grep @
"target": "@rules_license//licenses/spdx:Apache-2.0",
xls ๐ bazel query --color=no --noshow_progress --noshow_loading_progress --notool_deps --noimplicit_deps 'kind(cc_library, filter("^@", deps(//xls/dslx:interpreter_main)))' --output package | grep @ | wc -l
39
bazel 6.4.0
Linux penguin 6.1.64-09049-g010fe86d9eae #1 SMP PREEMPT_DYNAMIC Thu, 1 Feb 2024 01:25:43 +0000 x86_64 GNU/Linuxw
https://github.com/bazelbuild/rules_license/blob/main/rules/license.bzl#L112 prevents specifying license texts that are labels.
My project (Fuchsia) needs label support for:
To solve, once can implement the above check at the rule itself, instead of in the macro. e.g.
if not ctx.file.license_text.is_source:
fail("License file ", ctx.file.license_text.path, "must be a source file, not generated")
When running bazel build //examples/sboms:write_sbom_sbom
, the resulting JSON should look as follows:
[
{
"top_level_target": "//tools:write_sbom",
"dependencies": [
{
"target_under_license": "//tools:write_sbom",
"licenses": [
"//:license"
]
}
],
"licenses": [
{
"label": "//:license",
"bazel_package": "//",
"license_kinds": [
{
"target": "@//licenses/spdx:Apache-2.0",
"name": "Apache-2.0",
"conditions": []
}
],
"copyright_notice": "",
"package_name": "rules_license",
"package_url": "",
"package_version": "0.0.7",
"license_text": "LICENSE",
"used_by": [
"//tools:write_sbom"
]
}
],
"packages": [
{
"target": "//:package_info",
"bazel_package": "//",
"package_name": "rules_license",
"package_url": "",
"package_version": "0.0.7"
}
]
}
]
When running bazel build //examples/sboms:write_sbom_sbom
, the resulting JSON actually looks as follows:
[
{
"top_level_target": "//tools:write_sbom",
"dependencies": [
{
"target_under_license": "//tools:write_sbom",
"licenses": [
"//:license"
]
}
],
"licenses": [
{
"label": "//:license",
"bazel_package": "//",
"license_kinds": [
{
"target": "@//licenses/spdx:Apache-2.0",
"name": "Apache-2.0",
"conditions": []
}
],
"copyright_notice": "",
"package_name": "",
"package_url": "",
"package_version": "",
"license_text": "LICENSE",
"used_by": [
"//tools:write_sbom"
]
}
],
"packages": [
{
"target": "//:package_info",
"bazel_package": "//",
"package_name": "rules_license",
"package_url": "",
"package_version": "0.0.7"
},
{
"target": "//:package_info",
"bazel_package": "//",
"package_name": "rules_license",
"package_url": "",
"package_version": "0.0.7"
}
]
}
]
Specifically, the same package is included twice and the license doesn't contain package_name
and package_version
fields.
bazel build //examples/sboms:write_sbom_sbom
bazel-bin/examples/sboms/_write_sbom_sbom_licenses_info.json
The License json object is like:
[
{
"top_level_target": "//target:target",
"dependencies": [
{
"target_under_license": "@com_github_fmtlib_fmt//:lib",
"licenses": [
"@com_github_fmtlib_fmt//:license"
]
}
],
"licenses": [
{
"label": "@com_github_fmtlib_fmt//:license",
"rule": "@com_github_fmtlib_fmt//:license",
"license_kinds": [
{
"target": "@rules_license//licenses/spdx:MIT",
"name": "MIT",
"conditions": []
}
],
"copyright_notice": "",
"package_name": "fmtlib",
"package_url": "",
"package_version": "",
"license_text": "external/com_github_fmtlib_fmt/LICENSE.rst",
"used_by": [
"@com_github_fmtlib_fmt//:lib"
]
}
]
}
]
The licenses
argument of _do_licenses
at
rules_license/tools/checker_demo.py
Line 97 in e3bdc54
[
{
"label": "@com_github_fmtlib_fmt//:license",
"rule": "@com_github_fmtlib_fmt//:license",
"license_kinds": [
{
"target": "@rules_license//licenses/spdx:MIT",
"name": "MIT",
"conditions": []
}
],
"copyright_notice": "",
"package_name": "fmtlib",
"package_url": "",
"package_version": "",
"license_text": "external/com_github_fmtlib_fmt/LICENSE.rst",
"used_by": [
"@com_github_fmtlib_fmt//:lib"
]
}
]
but the unique_licenses
function try to get licenses
field from each item in the licenses list, so it will always be empty.
Plan here: Adding OSS license declarations to Bazel
Set up CI on buildkite.
I couldn't find any issue tracking this but please enlighten me if that is the case. Has there been any thoughts on creating a tool for gathering the license info and creating a BOM in the SPDX format? I am no expert, but to me it looks like the .json
produced by the license_used()
rule is not in the SPDX format.
Would that be in the scope of this library or something that users should be responsible for?
While trying out rules_license
for a ruleset I maintain, I noticed three problems that made it difficult for me to use:
license_kind
s are not publicly visible (fixed by #7).visibility
of license
targets (fixed by #8).http_archive
.Is rules_license
considered to be in a stable enough state that rulesets could already use it to add applicable_licenses
with well-known license_kinds
to their public targets? If so, it would be great if these three problems could be resolved.
exports_files
declarations should be traversed by the gathering aspect, although they are commonly used to import various licensed resources into build graphs.
Unfortunately, it looks like aspects don't traverse exports_files
targets at all, as these are not standard bazel rules. Furthermore, exports_files
also does not supports the applicable_licenses
attribute.
All examples work.
$ bazel test //examples/manifest:main_test
ERROR: /workspaces/rules_license/examples/manifest/BUILD:25:15: in cmd attribute of genrule rule //examples/manifest:gen_main_manifest: label '//examples/manifest:main_manifest' in $(locations) expression expands to no files. Since this rule was created by the macro 'android_binary', the error might have been caused by the macro implementation
ERROR: /workspaces/rules_license/examples/manifest/BUILD:25:15: Analysis of target '//examples/manifest:gen_main_manifest' failed
ERROR: Analysis of target '//examples/manifest:main_test' failed; build aborted:
INFO: Elapsed time: 0.154s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 6 targets configured)
ERROR: Couldn't start the build. Unable to run tests
bazel test //examples/manifest:main_test
Currently, the add_licenses.py
tool must be run with:
LC_ALL="en_US.UTF-8" admin/refresh_spdx/add_licenses.py
Instead, declare a py_binary
target in admin/refresh_spdx/BUILD
so the add_licenses.py
tool can be run with:
bazel run //admin/refresh_spdx:add_licenses
See PR review comment: #17 (comment)
Adding a license file should be optional.
When adding attributes for a package under the unlicense license it is missing in the package since it's not a condition of the license to keep the license file when distributing. We can not add just the license-file locally in our repo since glob cannot traverse different namespaces, from @package
-> @our_repo//workspace/package:LICENSE
.
The json output produced by licenses_info_to_json()
doesnt include include long_name
of License_kind
. However, we have a need for human-readable license names too.
The json output only contains only target
, name
, and condition
. However, we have a need for human-readable license names too.
This will be fixed in the 0.0.7 release. compliance.bzl
is mostly deprecated.
As the design doc says,
additional_info
: a map of strings to strings. This is a catch-all bucket for users to add custom metadata about this license instance. For example, if this is a paid license, it might indicate if it is site-wide or limited to a number of simultaneous instances. Nothing in this document refers to additional_info again.
No such attribute on the rule.
88789cc made TransitiveLicensesInfo private, but that is breaking downstream users of gather_licenses_info.
We need to do that in a soft transition and provide an alternative.
Need /licenses/generic:none, notice, unencumbered,
and maybe a few others.
We should be able to generate docs from the .bzl files.
That require using bz_library rules, but that would take a dependency on bazel-skylib, which impedes reuse.
Blocked on bazelbuild/bazel-skylib#127 (providing we resolve that by making bzl_library part of @bazel_tools)
The applicable_licenses
field was renamed inside of Bazel which breaks license collection on the latest 8.x releases of Bazel.
https://github.com/bazelbuild/bazel//commit/e54078fbe408a77a55cdaec7f0b8b35b19ae3010
When rules_jvm_external creates BUILD files, it should add package_info()
targets.
Typically, every BUILD file in a bazel module is under the license defined at //:license
. Since Bazel does not have inheritance from enclosing packages, we must usually add:
package(default_package_metadata=["//:license", "//:package_info"])
to every BUILD file. We should provide tools to help user maintain that invariant.
Initial thoughts:
bazel test
.We should provide //licenses/spdx:* for all of the SPDX identifiers.
That will allow package authors to more precisely declare what license the package is under.
In the future we could add tooling to generate SPDX file from a license() rule, but there are few consumers for that now. We can create an issue for that in the future if we want.
Currently, it is a manual process to check that the list of SPDX licenses represented by licenses/spdx/BUILD
are up-to-date.
Instead, add a CI test that checks what we have against the authoritative source and fails if we are more than some threshold behind.
See PR review comment: #17 (comment)
Let's update this repo to match https://github.com/bazel-contrib/rules-template
The release automation is completely performed in a standard, shared GitHub Actions reusable workflow.
As rule maintainers, this reduces the possibility we can make a mistake during a release, and improves security by allowing GitHub Actions to fully verify the release artifacts. In fact, we'll be able to add some SLSA-style attestation to the bazel-contrib/rules-template in a couple weeks.
It also reduces the maintenance burden for the repo to the minimum necessary, which is important for its long term health as individual engineers get pulled into other things.
Only the path to the license file is available in the JSON.
Have the actual content of the license file available in the JSON.
Currently, running add_licenses.py
requires an extra step to be run before it is executed:
wget https://github.com/spdx/license-list-data/raw/master/json/licenses.json
Also, add_licenses.py
expects the licenses.json
file to be in the correct place when it is executed, and it will fail if it is not found.
Instead, the call to download licenses.json
from Github should be incorporated into the add_licenses.py
script to remove the need for an extra step and room for user error
See PR review comment: #17 (comment)
There are too many things in the rules
package.
The expected use case is that everyone is going to by declaring a license
rule, but fewer people will be using those to produce license audit or SBOMs. Most of the people who do produce an SBOM are expected to check rules_license into their repo and modify or reuse as appropriate, so having a clear separation between the packages you modify and the ones you leave alone is helpful.
Some packages include an SBOM produced by publisher.
We should be able to represent that in package_info
Custom starlark rules should be able to return a LicensesInfo
provider in the event that they need to customize how licenses propagate. For example, some rules may need to propagate licenses through additional attributes which are unique to that rule. Others may for example want to filter the transitive license set based on the conditions on the license kinds.
gather_licenses_info
aspect adds a LicensesInfo
provider unconditionally, which results in an error if the target already has one. It should instead check whether the target already has a LicensesInfo
provider and return an empty provider list if it does.
It should be possible to create a license
that does not set a license_text
.
While it's certainly not a great idea to not have a license_text
for a license
, that's something that should be enforced (at the user's discretion) in compliance rules, rather than in the structure of the rule declarations. There are many situations where there isn't a practical alternative, e.g. downloading a python wheel where you know the SPDX ID of the license but the file isn't included in the zip file.
The macro wrapper sets license_text
to LICENSE
. This may be convenient in some cases, but at the same time that kind of invisible, implicit source file dependency is generally not the way most things work in bazel.
Rule Usage example:
load("@rules_license//rules:compliance.bzl", "check_license")
check_license(
name = "third_party_licenses",
license_texts = "third_party_licenses.txt",
report = "third_party_reports.txt",
deps = _ROOT_TARGETS,
)
ERROR: C:/users/dhmem/***/BUILD:116:14: in _check_license rule //third_party_licenses:
Traceback (most recent call last):
File "C:/users/dhmem/_bazel_dhmem/36t2vbit/external/rules_license/rules/compliance.bzl", line 33, column 24, in _check_license_impl
write_licenses_info(ctx, ctx.attr.deps, licenses_file)
File "C:/users/dhmem/_bazel_dhmem/36t2vbit/external/rules_license/rules/gather_licenses_info.bzl", line 144, column 50, in write_licenses_info
licenses.extend(licenses_info_to_json(dep[TransitiveLicensesInfo]))
File "C:/users/dhmem/_bazel_dhmem/36t2vbit/external/rules_license/rules/gather_licenses_info.bzl", line 245, column 58, in licenses_info_to_json
top_level_target = _strip_null_repo(licenses_info.target_under_license),
Error: 'TransitiveLicensesInfo' value has no field or method 'target_under_license'
Available attributes: deps, licenses, traces
ERROR: C:/users/dhmem/***/BUILD:116:14: Analysis of target '//third_party_licenses' failed
ERROR: Analysis of target '//third_party_licenses' failed; build aborted:
INFO: Elapsed time: 0.696s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (1 packages loaded, 0 targets configured)
Caused by an empty TransitiveLicensesInfo
:
DEBUG: C:/users/dhmem/_bazel_dhmem/36t2vbit/external/rules_license/rules/gather_licenses_info.bzl:243:10: licenses_info.target_under_license: struct(deps = depset([]), licenses = depset([]), traces = [])
And the empty TransitiveLicensesInfo
is from
rules_license/rules/licenses_core.bzl
Line 157 in d7eaaa6
Do we user a docs
branch or keep in main?
The current one is stale.
When you depend on a tool it should not be included in the license/metadata gathering.
We have the filtering function, but it is insufficient.
Some ideas:
gather_licenses_info_and_write
includes unnexpected dependencies and licenses.There should be a way to filter these targets out
gather_licenses_info_and_write
aspect to a pkg_rpm()
target like:pkg_rpm(
name = "xyz-rpm",
srcs = [...]
...
)
, which transitively consumes some deps like postgresql, javax-mail etc.
2. The JSON output produces by the aspect reports @rules_pkg//pkg:make_rpm
as a dependency target in addition to correctly identifying postgresql, javax-mail etc. There should be a way to filter out these targets as dependencies and their associated licenses.
Sample output can be found here:
https://gist.github.com/rtabassum/b9de682c2939f7e0b882d86f8686ed12#file-xyz-licenses-json
Trying to apply the license collection mechanism outlined in the examples/manifest directory onto another Bazel based project fails at load("@rules_license//rules:compliance.bzl", "manifest")
in the WORKSPACE file with error Error: file '@rules_license//rules:compliance.bzl' does not contain symbol 'manifest'
.
Issuing a load("@rules_license//rules:license_kind.bzl", "license_kind")
in the same WORKSPACE file works fine, though, so the rules_license http_archive was found and properly loaded.
That same load(...) works when building inside the examples/manifest directory, which makes me suspect the visibility is somewhat wrong or is the examples/manifest something that is meant to be looked at when trying out license collection? Is the proper way to use rules_license in ones own project described somewhere in a way that a novice Bazel user could evaluate its functionality?
Today, developers must take care to include sources in the filegroup(name = "standard_package")
targets which are sprinkled around the repository.
In the best case, omitting one will be caught by automated testing in the repo. However,
I also claim there is no benefit to having Bazel build the release artifact. git archive
is completely sufficient for this task (including pruning directories that are unnecessary in the release artifact and which make it large)
Making this change would simplify the repository so that it's more feasible for us to maintain it with a very small effort.
After updating //licenses/spdx/BUILD, all the license_kind definitions should be in alphabetical order by name.
After each run, new items are added in alphabetically ordered chunks. That is ugly.
Implementation question: Do we sort with LC_ALL=C or use unicode "human readable" ordering?
Towards the second point, we want to be able to minimize the number of places where we have to rewrite @rules_license
to the //path/to/my/rules_license
if we import the package into SCM. Test data should not be transformed, and the test itself might have a single constant within it (or injected to the test BUILD target) to provide the root of the tree.
LicensesInfo
/ LicenseInfo
will be collected by reviewing all attributes of objects passed in to the gather_licenses_info
aspect.
The list of attributes that are inspected for LicenseInfo
/ LicensesInfo
does not gather licenses for all third party libraries included in an android app binary.
gather_licenses_info
aspect to collect license information for your deps
Note: the same way as shown in this docstring
We found including exports
allows us to find significantly more licenses when building an android binary.
Version: 0.0.3
Platform: Android
I just wanted to move our requirements from a comment on a closed PR (#6 (comment)) to the bugtracker:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.