Giter VIP home page Giter VIP logo

platforms's Introduction

This repository houses all canonical constraint_setting()s, constraint_value()s and platform()s that are universally useful across languages and Bazel projects.

For questions or concerns please email [email protected].

Motivation

Constraints must be carefully organized to avoid fragmentation. If two different declarations for, say, cpu=x86_64 were to exist at the same time then select() statements and toolchain selection would stop working across languages and projects.

Process

This repository only includes truly ubiquitous constraints.

Most common constraints -- that is settings and values that can be used across projects - fall into specific areas like "Apple" or "Java". These are declared in those areas' respective repositories.

A very few constraints (such as OS and cpu) are relevant for essentially all projects across all areas. These are what this repository is for.

Adding a canonical constraint value

To add a new canonical constraint value, prepare a PR adding it to that the appropriate BUILD file.

Note that even global constraint values are typically area values. For example, ios is an area value for the global setting os but belongs in the apple area. For the PR's reviewer(s) choose an owner of this repository plus an owner of the area repository this references..

A constraint value should be:

  • semantically clear, particularly in its distinctions from other values of the same setting

  • well-named: consistent with existing values of the same setting and easy to understand at usage sites

  • well-documented

Remember that this value will apply for the entire Bazel community and its semantics will be difficult to impossible to change once it starts being used.

Adding a canonical constraint setting

New canonical constraint settings should be rare and well-justified.

To add a new setting, prepare a design document according to Bazel's design review process. This document should explain the need for a new setting and why it belongs here vs. area-specific repositories. It should clearly explain semantics, initial values, and criteria for adding new values.

Once the design is approved prepare a PR for the actual change. If any values are area-specific, include the area repositories' owners as reviewers.

Private changes to global constraints

If you'd like to experiment with changes to global settings or values, you can fork this repo for experimental purposes. But in the interest of community health and interoperability please don't share your changes with anyone not involved with the experiment. For wider distribution, submit a proper change here.

Note that you can declare constraint_values in your own repo that are members of the global constraint_settings. This lets you "extend" global settings within the confines of your own project. But don't do this if you expect other projects to use these changes - this can easily lead to fragmentation conflicts.

If you need custom constaint_settings, just declare them in your own repo. They are, by definition, not global.

If you really need a permanent global change and it isn't design-approved for this repo, start a thread on GitHub to discuss options.

platforms's People

Contributors

aiuto avatar allevato avatar cfrantz avatar comius avatar fweikert avatar googlewalt avatar gregestren avatar hlopko avatar iirina avatar katre avatar kmoffett avatar lberki avatar lizan avatar mangini avatar maskray avatar mboes avatar merore avatar meteorcloudy avatar mtribiere avatar philsc avatar philwo avatar piotrsikora avatar quantamhd avatar sergiocampama avatar silvergasp avatar tevo45 avatar thii avatar wyverald avatar x1ddos avatar znley avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

platforms's Issues

i386 vs x86_32

Currently, there are two separate cpu constraint values x86_32 and i386. The intended use is not entirely clear to me.
As I understand it, "x86" is the whole family of instruction set architectures (also known as Intel Architecture "IA-32"), and, technically, "i386" (also known as 80386) is the first processor in the line of this family.
However, the two terms are frequently used interchangeably; for example, the Debian port supporting all 32 bit x86 CPUs is called "i386" (see https://www.debian.org/ports/).

Is this i386 cpu constraint value really intended to refer specifically to (the instruction set of the) original 80386 Intel processor? Or does it rather refer to the entire family, just like x86_32?
If it's the former, would it make sense to add a big warning note to indicate that you probably don't want to use this?
If it's the latter, would it make sense to change i386 to be merely an alias for x86_32?

Constraints for CPU features?

Hi. Would it be possible to detect CPU features using something
like https://github.com/google/cpu_features/
and provide them as platform constraints?

This could help in enabling certain compiler flags and instruction set
architectures for targets.

What needs to be done so that something like this would work
with Bazel?

How to use the wasm32 CPU target

#12 adds the wasm32 CPU. But when I try to use it in my project, I get:

ERROR: /home/huyage/dev/wasm/wasmbench/platforms/BUILD.bazel:9:9: no such target '@platforms//cpu:wasm32': target 'wasm32' not declared in package 'cpu' defined by /home/huyage/.cache/bazel/_bazel_huyage/82c2da1739c371e7ca0d69f58318b066/external/platforms/cpu/BUILD and referenced by '//platforms:wasm32'

I'm on the latest Bazel version using bazelisk.

Define a value for "no OS"?

Hello!

I'm working on embedded code which does not run with any OS underneath (on "bare metal"). My initial thought was to specifiy "OS=none" or something like that. What do you think of adding such a constant?

One issue I see is that it looks like it is not really a constraint, rather the opposite thereof. On the other hand, bare metal means direct access to device registers and similar things that the OS prevents normally. Interestingly, compiling an OS kernel itself would be one example of a program that requires this.

Thoughts? Ideas? Concerns?

Uli

Constraints for ABI

Would it be possible to introduce @platforms//abi which contains constraints for common ABI definitions of platform triples?

In rules_rust we currently have issues with users wanting to target platforms with shared CPU and system and unique ABI but are unable to do so as there's no constraints uniquely identifying them. Users are forced to define custom constraints and redefine toolchains to get around errors caused by this ambiguity. I think it would be very beneficial to introduce ABI constraints here so that rules_rust and other rules can share constraints, enabling rule maintainers to appropriately constrain toolchains so users only have to setup platform definitions.

Implement Standard Platform Transitions

This is a tracking issue for the implementation of Standard Platform Transitions.

The steps required are:

The implementation can be based off of Aspect's platform_transition_binary and Skylib's native_binary.

Note that the platform_data implementation should:

  • Make sure that the new files are executable
  • Make a copy, not just a symlink
  • Work with rules such as cc_binary, py_binary, and sh_binary.

Define a distribution constraint setting

A common trick with Bazel rules is to download prebuilt SDK's and use that as toolchains. However, it is a mistake to assume that keying which SDK to download on the @bazel_tools//platforms:os value will work in every case. SDK's are seldom self-contained and often assume a specific glibc, or filesystem layout. On Linux, not all distributions are alike. On macOS, not all versions are alike. So we need finer grained platform info.

One idea would be to add a generic distribution constraint setting, in addition to the existing os and cpu constraint settings. The distribution setting could be used to choose the SDK format as well. For example, while the default might be a binary distribution as commonly distributed by upstream (official Go SDK's, official GHC binary distributions, etc), alternatives for obtaining a working toolchain might be to use Snappy, AppImage, Nix or Flatpak, depending on what is available or what the user prefers.

Not all toolchain distribution methods work on all platforms. Binary distributions seldom work on NixOS because the dynamic linker is not at a known location. Conversely, using Nix, be it on macOS or Linux, requires first that Nix be installed (same from Snappy etc). One might argue that Flatpak, Nix or Snappy should be an os constraint value. But that's very inconvenient. For example, on macOS dynamic libraries are called libfoo.dylib rather than libfoo.so. Rules want to deal with macOS specifics, while also letting the user choose among several possible sources fo toolchains. To allow that with just one constraint setting, we'd have to define many constraint values, like osx, osx_nix, linux, linux_nix etc.

Nix is available as a distribution method both on Linux and on macOS. Flatpak, Snappy, AppImage and the like are for the moment Linux-specific. But could well likewise be multi-platform in the future. Defining a new constraint_setting would avoid the combinatorial explosion in constraint values that would otherwise be needed if we continue having just @bazel_tools//platforms:os.

Please list names of CODEOWNERS

Please list names (and, preferably, email addresses) of the people in CODEOWNERS. This will allow proper copyright attribution when this code is outside GitHub.

Processors and Environments

Hi all, trust all is well!

I want to ask you're help with something. Have been googling around for a related set of problems, but can't seem to find a solution for the problem. I don't know if this is the correct place to post this, if not, kindly provide me a location where I can.

I'm in the process of developing a small OS. While doing the programming tasks have been progressing quite nicely, I'm having a problem with doing the cross-platform stuff inside Bazel.

The idea I have is, that the OS can be compiled for a specific device, say, the Pixel phone. The device has a very specific set of features, such as an ARM processor, it's 64-bit, and the CPU also has bi-endian.

No the problem comes in when specifying the toolchains. I understand that I can specify constraints for all things (CPU Architecture, CPU Sub-Architecture, Bit Size, Endianess). It's just the explosion of variations that comes with it. For the CPU architecture, you have possibly x86 and ARM, and then for the sub-architectures you have on the x86 side Sandy Bridge, Knights Landing, etc.

Bit sizes and endianess aside, there's also different "environments", like the firmware (UEFI, BIOS, etc.), kernel-mode code, driver code, user-mode code, etc.

So basically, is there a way to use the platforms workspace for this, can it be cleanly mapped? I know I will have to define a few constraints for the extra features.

Also, how would I manage the various combinations of these items, like UEFI on x86 Sandy Bridge, and U-Boot on ARM Corext-A53 for example? I've been looking at possible ways of doing inheritance inside Starlark for the C++ toolchain rules, but can't seem to find a nice solution, always ending up with a lot of duplication.

Repository `@rules_license` is not defined

I'm not using bzlmod so I'm getting an error due to not defining rules_license in my WORKSPACE file and a recent commit of this repo and use of @platforms//:incompatible. Should there be a dependencies macro as mentioned here or is bzlmod the only way going forward?

Issue specifying linux platform constraint to bazel workspace

Currently I am attempting to build a starter grpc project with bazel[version 6.3.2]. Upon building, I get the following error, which I believe requires me to add the platform constraints[I believe the default platform is windows, while I am using an ubuntu os]:

ERROR: /home/arpit/.cache/bazel/_bazel_arpit/f196a2da91eeaec415a1459ab74281d6/external/bazel_tools/platforms/BUILD:89:6: in alias rule @bazel_tools//platforms:windows: Constraints from @bazel_tools//platforms have been removed. Please use constraints from @platforms repository embedded in Bazel, or preferably declare dependency on https://github.com/bazelbuild/platforms. See https://github.com/bazelbuild/bazel/issues/8622 for details.

Thus I added the following to my project

inside WORKSPACE

git_repository(
    name = "platforms",
    commit = "33a3b209f94856193266871b1545054afb90bb28",
    remote = "https://github.com/bazelbuild/platforms",
)

I then created a linux_platform.bzl file inside the workspace:

load("@bazel_tools//tools/build_defs:platforms/linux.bzl", "platform")

platform(
    name = "linux",
    constraint_values = {
        "@platforms//os:linux": 1,
    },
)

and then linked the linux_platform file inside my project BUILD file to force the build for linux os:

load("//:linux_platform.bzl", "linux")

However this fails, with the following error:

ERROR: Skipping ':all': while parsing ':all': error loading package 'projects/python_folder/grpc': at /home/arpit/bazel_ws/workspace-template/linux_platform.bzl:1:6: cannot load '@bazel_tools//tools/build_defs:platforms/linux.bzl': no such file

I am new to bazel environment [previously have been using catkin/cmake], so I apologize for the noob question. What am I missing here, I see that linux.bzl is not part of the platforms repo, but I thought should it would built with the os folder. How do I provide the linux constraint ?

Should we make a branch per Bazel LTS branch?

There would be one branch per LTS fork - 3.x, 4.x. main would be mirrored to Bazel at head.

Pro:

  • it seems like the easiest thing to manage conceptually.

Con:

  • are we really likely to be adding such drastic new concepts that any changes can not be backported into older Bazel branches?

How to reference the platform repository.

Hi All,

I am new to use bazel build system. I installed the latest bazelisk and bazel with 6.0.0. However when I tried to build fleetbench the following messages were pop out.
bazel run -c opt fleetbench/swissmap:cold_swissmap_benchmark
ERROR: /root/.cache/bazel/_bazel_root/0bce1989468318c371f4348e6ac4d902/external/bazel_tools/platforms/BUILD:89:6: in alias rule @bazel_tools//platforms:windows: Constraints from @bazel_tools//platforms have been removed. Please use constraints from @platforms repository embedded in Bazel, or preferably declare dependency on https://github.com/bazelbuild/platforms. See bazelbuild/bazel#8622 for details.
ERROR: /root/.cache/bazel/_bazel_root/0bce1989468318c371f4348e6ac4d902/external/bazel_tools/platforms/BUILD:89:6: Analysis of target '@bazel_tools//platforms:windows' failed
ERROR: /home/nvidia/walter/fleetbench/fleetbench/swissmap/BUILD:30:28: While resolving toolchains for target //fleetbench/swissmap:cold_swissmap_benchmark: invalid registered toolchain '@bazel_skylib//toolchains/unittest:cmd_toolchain':
ERROR: Analysis of target '//fleetbench/swissmap:cold_swissmap_benchmark' failed; build aborted:
INFO: Elapsed time: 1.363s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (33 packages loaded, 85 targets configured)
ERROR: Build failed. Not running target

I had checked the issue 8622. But I didn't get any clue how to declare dependency on the platform repository. Could someone give me a hint? Thanks in advance.

Walter

Platform Commonalities: e.g. Apple, POSIX, etc.

Hi wonderful Bazel Platforms folks!

First and foremost, thanks for working to create something centralized and great here. It's so nice to have official support for modeling cross-platform code with platform-specific backends (among other goodies).

I'd like to float an idea for making official something I find myself rebuilding pretty much every time I use platforms.

Lots of platforms (err constraint_values) have considerable code in common. It can be nice to select on this commonality. Some examples are: Apple Platforms. POSIX/UNIX-compliant platforms. Etc.

It's not hard to model these (example below), but it'd be awesome to have them in the @platforms repo for everyone to use!

load("@bazel_skylib//lib:selects.bzl", "selects")
selects.config_setting_group(
    name = "Apple",
    visibility = ["//visibility:public"],
    match_any = ["@platforms//os:macos", "@platforms//os:ios", "@platforms//os:tvos", "@platforms//os:watchos"],
)

Happy to contribute of course--and would have made this a PR--except that I'm not sure quite how this repo is being automagically integrated into Bazel proper. Therefore I wasn't sure how to properly get skylib.

Thanks again, additionally for your consideration!
Chris
(ex-Googler)

platform_data syntax error

The current definition of platform_data has a syntax error: it has def _target_platform_transition_impl(attr), but should be def _target_platform_transition_impl(settings, attr).

This is why we need to add tests. I'll look into setting that up this week.

Please create a release

The contents of this repository are useful (and should be part of the upcoming Debian Bazel package) but using them would be easier if there was an actual release to refer to downstream. Even a 0.0.1 would work if this is not at a 1.0 level yet. Thanks!

MacOS being detected as multiple OSes

So I have a BUILD file that looks like this


package(default_visibility = ["//visibility:public"])

config_setting(
    name = "windows",
    constraint_values = [
        "@platforms//os:windows",
    ],
    visibility = ["//visibility:public"],
)

config_setting(
    name = "mac",
    constraint_values = [
        "@platforms//os:macos",
    ],
    visibility = ["//visibility:public"],
)

config_setting(
   name = "linux",
   constraint_values = [
      "@platforms//os:linux",
   ],
   visibility = ["//visibility:public"],
)

cc_library(
   name = "db_drivers",
   srcs = select({ 
      "windows": glob(["**/*.a"]),
      "mac": ["liblmdb/midl.c", "liblmdb/mdb.c"],
      "linux": glob(["**/*.a"]),
   }),
   hdrs = glob(["**/liblmdb/*.h"]),
)

This is meant to be an external repository that I am defining a rule for. But when I run bazel query on the path to the library I have that depends on this I get the following.

@db_drivers//:liblmdb/liblmdb.a
@db_drivers//:liblmdb/lmdb.h
@db_drivers//:liblmdb/mdb.c
@db_drivers//:liblmdb/midl.c
@db_drivers//:liblmdb/midl.h

As you can see it is running the glob that is picking up the .a file even though that should only happen on mac. When I remove the linux part of the select it works and I don't pull in the .a. What am I doing wrong here?

[question] bazel platforms for c/c++ how to distinguish the toolchain while they are same cpu and os

HI ALL:
I have encountered a problem: our c/c++ project need to cross compile to different arm boards, and they are all aarch64 & linxu, but with three different toolchains.
The platforms has only os and cpu two options to decide a toolchain。Arm I right?
So how do I add another option to approach this ? Such as board_type?
platform(
name = "platfomr_board_A",
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:aarch64",
":board_type_A",
],
)

platform(
name = "platfomr_board_B",
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:aarch64",
":board_type_B",
],
)

platform(
name = "platfomr_board_C",
constraint_values = [
"@platforms//os:linux",
"@platforms//cpu:aarch64",
":board_type_C",
],
)

and the run in the cmd line: bazel build --platfomrs=//:platfomr_board_B //...

Add a set of co-proccesor constraint_setting (feature request)

This is a similar suggestion to #13 but more generic and could include other secondary processors. This could include fpu's, gpgpu's, ai proccesors, simd extensions, dsp extensions, fpga's etc.

An example of this might look like;

coproccesor/BUILD

licenses(["notice"])
package(
    default_visibility = ["//visibility:public"],
)

constraint_setting(name = "fpu")

# Cortex-M4 optional fpu
constraint_value(
    name = "VFPv4_SP", 
    constraint_setting = ":fpu",
)

# Cortex-M7 optional fpu
constraint_value(
    name = "VFPv5_D16", 
    constraint_setting = ":fpu",
)

constraint_setting(name="dsp")

constraint_value(
    name = "cortex_m_dsp", 
    constraint_setting = ":dsp",
)

constraint_setting(name="ai")

constraint_value(
    name = "coral_edge_tpu", 
    constraint_setting = ":ai",
)

constraint_setting(name="gpgpu")

constraint_value(
    name = "amd-opencl", 
    constraint_setting = ":gpgpu",
)

constraint_value(
    name = "nvidia-cuda", 
    constraint_setting = ":gpgpu",
)

There is one caveat with using bazel platforms/constraint settings here and that is if you had a device that could support two constraint_values for a given constraint_setting. Say that you put a "AMD" and a "nvidia" gpgpu into a device you couldn't specify multiple values for the 'gpgpu' constraint_setting. This could be a common recurring problem, specifically with integrated+dedicated gpgpu devices.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.