bazelbuild / platforms Goto Github PK
View Code? Open in Web Editor NEWConstraint values for specifying platforms and toolchains
License: Apache License 2.0
Constraint values for specifying platforms and toolchains
License: Apache License 2.0
A common trick with Bazel rules is to download prebuilt SDK's and use that as toolchains. However, it is a mistake to assume that keying which SDK to download on the @bazel_tools//platforms:os
value will work in every case. SDK's are seldom self-contained and often assume a specific glibc, or filesystem layout. On Linux, not all distributions are alike. On macOS, not all versions are alike. So we need finer grained platform info.
One idea would be to add a generic distribution
constraint setting, in addition to the existing os
and cpu
constraint settings. The distribution
setting could be used to choose the SDK format as well. For example, while the default might be a binary distribution as commonly distributed by upstream (official Go SDK's, official GHC binary distributions, etc), alternatives for obtaining a working toolchain might be to use Snappy, AppImage, Nix or Flatpak, depending on what is available or what the user prefers.
Not all toolchain distribution methods work on all platforms. Binary distributions seldom work on NixOS because the dynamic linker is not at a known location. Conversely, using Nix, be it on macOS or Linux, requires first that Nix be installed (same from Snappy etc). One might argue that Flatpak, Nix or Snappy should be an os
constraint value. But that's very inconvenient. For example, on macOS dynamic libraries are called libfoo.dylib
rather than libfoo.so
. Rules want to deal with macOS specifics, while also letting the user choose among several possible sources fo toolchains. To allow that with just one constraint setting, we'd have to define many constraint values, like osx
, osx_nix
, linux
, linux_nix
etc.
Nix is available as a distribution method both on Linux and on macOS. Flatpak, Snappy, AppImage and the like are for the moment Linux-specific. But could well likewise be multi-platform in the future. Defining a new constraint_setting
would avoid the combinatorial explosion in constraint values that would otherwise be needed if we continue having just @bazel_tools//platforms:os
.
Hi all, trust all is well!
I want to ask you're help with something. Have been googling around for a related set of problems, but can't seem to find a solution for the problem. I don't know if this is the correct place to post this, if not, kindly provide me a location where I can.
I'm in the process of developing a small OS. While doing the programming tasks have been progressing quite nicely, I'm having a problem with doing the cross-platform stuff inside Bazel.
The idea I have is, that the OS can be compiled for a specific device, say, the Pixel phone. The device has a very specific set of features, such as an ARM processor, it's 64-bit, and the CPU also has bi-endian.
No the problem comes in when specifying the toolchains. I understand that I can specify constraints for all things (CPU Architecture, CPU Sub-Architecture, Bit Size, Endianess). It's just the explosion of variations that comes with it. For the CPU architecture, you have possibly x86 and ARM, and then for the sub-architectures you have on the x86 side Sandy Bridge, Knights Landing, etc.
Bit sizes and endianess aside, there's also different "environments", like the firmware (UEFI, BIOS, etc.), kernel-mode code, driver code, user-mode code, etc.
So basically, is there a way to use the platforms
workspace for this, can it be cleanly mapped? I know I will have to define a few constraints for the extra features.
Also, how would I manage the various combinations of these items, like UEFI on x86 Sandy Bridge, and U-Boot on ARM Corext-A53 for example? I've been looking at possible ways of doing inheritance inside Starlark for the C++ toolchain rules, but can't seem to find a nice solution, always ending up with a lot of duplication.
"aarch64" is the perferred name and used prevailingly on Linux. See https://nickdesaulniers.github.io/blog/2023/03/10/disambiguating-arm/
Apple and Windows platforms unfortunately use "arm64". They can still use the alias.
Currently I am attempting to build a starter grpc project with bazel[version 6.3.2]. Upon building, I get the following error, which I believe requires me to add the platform constraints[I believe the default platform is windows, while I am using an ubuntu os]:
ERROR: /home/arpit/.cache/bazel/_bazel_arpit/f196a2da91eeaec415a1459ab74281d6/external/bazel_tools/platforms/BUILD:89:6: in alias rule @bazel_tools//platforms:windows: Constraints from @bazel_tools//platforms have been removed. Please use constraints from @platforms repository embedded in Bazel, or preferably declare dependency on https://github.com/bazelbuild/platforms. See https://github.com/bazelbuild/bazel/issues/8622 for details.
Thus I added the following to my project
inside WORKSPACE
git_repository(
name = "platforms",
commit = "33a3b209f94856193266871b1545054afb90bb28",
remote = "https://github.com/bazelbuild/platforms",
)
I then created a linux_platform.bzl file inside the workspace:
load("@bazel_tools//tools/build_defs:platforms/linux.bzl", "platform")
platform(
name = "linux",
constraint_values = {
"@platforms//os:linux": 1,
},
)
and then linked the linux_platform file inside my project BUILD file to force the build for linux os:
load("//:linux_platform.bzl", "linux")
However this fails, with the following error:
ERROR: Skipping ':all': while parsing ':all': error loading package 'projects/python_folder/grpc': at /home/arpit/bazel_ws/workspace-template/linux_platform.bzl:1:6: cannot load '@bazel_tools//tools/build_defs:platforms/linux.bzl': no such file
I am new to bazel environment [previously have been using catkin/cmake], so I apologize for the noob question. What am I missing here, I see that linux.bzl is not part of the platforms repo, but I thought should it would built with the os folder. How do I provide the linux constraint ?
This is a tracking issue for the implementation of Standard Platform Transitions.
The steps required are:
platform_data
change_platform
(for a static platform)change_platform
(with a platform
attribute)The implementation can be based off of Aspect's platform_transition_binary
and Skylib's native_binary
.
Note that the platform_data
implementation should:
cc_binary
, py_binary
, and sh_binary
.Hello!
I'm working on embedded code which does not run with any OS underneath (on "bare metal"). My initial thought was to specifiy "OS=none" or something like that. What do you think of adding such a constant?
One issue I see is that it looks like it is not really a constraint, rather the opposite thereof. On the other hand, bare metal means direct access to device registers and similar things that the OS prevents normally. Interestingly, compiling an OS kernel itself would be one example of a program that requires this.
Thoughts? Ideas? Concerns?
Uli
HI ALL:
I have encountered a problem: our c/c++ project need to cross compile to different arm boards, and they are all aarch64 & linxu, but with three different toolchains.
The platforms has only os and cpu two options to decide a toolchain。Arm I right?
So how do I add another option to approach this ? Such as board_type?
platform(
name = "platfomr_board_A",
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:aarch64",
":board_type_A",
],
)
platform(
name = "platfomr_board_B",
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:aarch64",
":board_type_B",
],
)
platform(
name = "platfomr_board_C",
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:aarch64",
":board_type_C",
],
)
and the run in the cmd line: bazel build --platfomrs=//:platfomr_board_B //...
There would be one branch per LTS fork - 3.x, 4.x. main would be mirrored to Bazel at head.
Pro:
Con:
Is //cpu:ppc
the equivalent of PowerPC 64bit?
Do we need a new constraint or an alias to ppc
?
Please list names (and, preferably, email addresses) of the people in CODEOWNERS. This will allow proper copyright attribution when this code is outside GitHub.
Would it be possible to introduce @platforms//abi
which contains constraints for common ABI definitions of platform triples?
In rules_rust
we currently have issues with users wanting to target platforms with shared CPU and system and unique ABI but are unable to do so as there's no constraints uniquely identifying them. Users are forced to define custom constraints and redefine toolchains to get around errors caused by this ambiguity. I think it would be very beneficial to introduce ABI constraints here so that rules_rust
and other rules can share constraints, enabling rule maintainers to appropriately constrain toolchains so users only have to setup platform definitions.
The contents of this repository are useful (and should be part of the upcoming Debian Bazel package) but using them would be easier if there was an actual release to refer to downstream. Even a 0.0.1 would work if this is not at a 1.0 level yet. Thanks!
This is a similar suggestion to #13 but more generic and could include other secondary processors. This could include fpu's, gpgpu's, ai proccesors, simd extensions, dsp extensions, fpga's etc.
An example of this might look like;
coproccesor/BUILD
licenses(["notice"])
package(
default_visibility = ["//visibility:public"],
)
constraint_setting(name = "fpu")
# Cortex-M4 optional fpu
constraint_value(
name = "VFPv4_SP",
constraint_setting = ":fpu",
)
# Cortex-M7 optional fpu
constraint_value(
name = "VFPv5_D16",
constraint_setting = ":fpu",
)
constraint_setting(name="dsp")
constraint_value(
name = "cortex_m_dsp",
constraint_setting = ":dsp",
)
constraint_setting(name="ai")
constraint_value(
name = "coral_edge_tpu",
constraint_setting = ":ai",
)
constraint_setting(name="gpgpu")
constraint_value(
name = "amd-opencl",
constraint_setting = ":gpgpu",
)
constraint_value(
name = "nvidia-cuda",
constraint_setting = ":gpgpu",
)
There is one caveat with using bazel platforms/constraint settings here and that is if you had a device that could support two constraint_values for a given constraint_setting. Say that you put a "AMD" and a "nvidia" gpgpu into a device you couldn't specify multiple values for the 'gpgpu' constraint_setting. This could be a common recurring problem, specifically with integrated+dedicated gpgpu devices.
The current definition of platform_data
has a syntax error: it has def _target_platform_transition_impl(attr)
, but should be def _target_platform_transition_impl(settings, attr)
.
This is why we need to add tests. I'll look into setting that up this week.
I'm not using bzlmod
so I'm getting an error due to not defining rules_license
in my WORKSPACE
file and a recent commit of this repo and use of @platforms//:incompatible
. Should there be a dependencies macro as mentioned here or is bzlmod
the only way going forward?
Currently, there are two separate cpu constraint values x86_32
and i386
. The intended use is not entirely clear to me.
As I understand it, "x86" is the whole family of instruction set architectures (also known as Intel Architecture "IA-32"), and, technically, "i386" (also known as 80386) is the first processor in the line of this family.
However, the two terms are frequently used interchangeably; for example, the Debian port supporting all 32 bit x86 CPUs is called "i386" (see https://www.debian.org/ports/).
Is this i386
cpu constraint value really intended to refer specifically to (the instruction set of the) original 80386 Intel processor? Or does it rather refer to the entire family, just like x86_32
?
If it's the former, would it make sense to add a big warning note to indicate that you probably don't want to use this?
If it's the latter, would it make sense to change i386
to be merely an alias for x86_32
?
I'm waiting a little bit for @platforms//os:emscripten
😄
Hi. Would it be possible to detect CPU features using something
like https://github.com/google/cpu_features/
and provide them as platform constraints?
This could help in enabling certain compiler flags and instruction set
architectures for targets.
What needs to be done so that something like this would work
with Bazel?
So I have a BUILD
file that looks like this
package(default_visibility = ["//visibility:public"])
config_setting(
name = "windows",
constraint_values = [
"@platforms//os:windows",
],
visibility = ["//visibility:public"],
)
config_setting(
name = "mac",
constraint_values = [
"@platforms//os:macos",
],
visibility = ["//visibility:public"],
)
config_setting(
name = "linux",
constraint_values = [
"@platforms//os:linux",
],
visibility = ["//visibility:public"],
)
cc_library(
name = "db_drivers",
srcs = select({
"windows": glob(["**/*.a"]),
"mac": ["liblmdb/midl.c", "liblmdb/mdb.c"],
"linux": glob(["**/*.a"]),
}),
hdrs = glob(["**/liblmdb/*.h"]),
)
This is meant to be an external repository that I am defining a rule for. But when I run bazel query
on the path to the library I have that depends on this I get the following.
@db_drivers//:liblmdb/liblmdb.a
@db_drivers//:liblmdb/lmdb.h
@db_drivers//:liblmdb/mdb.c
@db_drivers//:liblmdb/midl.c
@db_drivers//:liblmdb/midl.h
As you can see it is running the glob
that is picking up the .a
file even though that should only happen on mac. When I remove the linux
part of the select it works and I don't pull in the .a
. What am I doing wrong here?
Hi wonderful Bazel Platforms folks!
First and foremost, thanks for working to create something centralized and great here. It's so nice to have official support for modeling cross-platform code with platform-specific backends (among other goodies).
I'd like to float an idea for making official something I find myself rebuilding pretty much every time I use platforms.
Lots of platforms (err constraint_values) have considerable code in common. It can be nice to select on this commonality. Some examples are: Apple Platforms. POSIX/UNIX-compliant platforms. Etc.
It's not hard to model these (example below), but it'd be awesome to have them in the @platforms repo for everyone to use!
load("@bazel_skylib//lib:selects.bzl", "selects")
selects.config_setting_group(
name = "Apple",
visibility = ["//visibility:public"],
match_any = ["@platforms//os:macos", "@platforms//os:ios", "@platforms//os:tvos", "@platforms//os:watchos"],
)
Happy to contribute of course--and would have made this a PR--except that I'm not sure quite how this repo is being automagically integrated into Bazel proper. Therefore I wasn't sure how to properly get skylib.
Thanks again, additionally for your consideration!
Chris
(ex-Googler)
Hi All,
I am new to use bazel build system. I installed the latest bazelisk and bazel with 6.0.0. However when I tried to build fleetbench the following messages were pop out.
bazel run -c opt fleetbench/swissmap:cold_swissmap_benchmark
ERROR: /root/.cache/bazel/_bazel_root/0bce1989468318c371f4348e6ac4d902/external/bazel_tools/platforms/BUILD:89:6: in alias rule @bazel_tools//platforms:windows: Constraints from @bazel_tools//platforms have been removed. Please use constraints from @platforms repository embedded in Bazel, or preferably declare dependency on https://github.com/bazelbuild/platforms. See bazelbuild/bazel#8622 for details.
ERROR: /root/.cache/bazel/_bazel_root/0bce1989468318c371f4348e6ac4d902/external/bazel_tools/platforms/BUILD:89:6: Analysis of target '@bazel_tools//platforms:windows' failed
ERROR: /home/nvidia/walter/fleetbench/fleetbench/swissmap/BUILD:30:28: While resolving toolchains for target //fleetbench/swissmap:cold_swissmap_benchmark: invalid registered toolchain '@bazel_skylib//toolchains/unittest:cmd_toolchain':
ERROR: Analysis of target '//fleetbench/swissmap:cold_swissmap_benchmark' failed; build aborted:
INFO: Elapsed time: 1.363s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (33 packages loaded, 85 targets configured)
ERROR: Build failed. Not running target
I had checked the issue 8622. But I didn't get any clue how to declare dependency on the platform repository. Could someone give me a hint? Thanks in advance.
Walter
#12 adds the wasm32 CPU. But when I try to use it in my project, I get:
ERROR: /home/huyage/dev/wasm/wasmbench/platforms/BUILD.bazel:9:9: no such target '@platforms//cpu:wasm32': target 'wasm32' not declared in package 'cpu' defined by /home/huyage/.cache/bazel/_bazel_huyage/82c2da1739c371e7ca0d69f58318b066/external/platforms/cpu/BUILD and referenced by '//platforms:wasm32'
I'm on the latest Bazel version using bazelisk.
There are a number of widely used config_setting
targets in @bazel_tools//src/conditions
, that should probably actually be in the @platforms
repository.
We should investigate migrating these to simplify the setup.
platforms
0.0.6 was published on Aug 27,
However it is not published to the central bazel repo.
ERROR: Error computing the main repository mapping: module not found in registries: [email protected]
See - https://github.com/bazelbuild/bazel-central-registry/tree/main/modules/platforms
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.