bazelbuild / rules_perl Goto Github PK

The perl binaries included in the toolchains are not static binaries. Instead, they link against a number of libraries:

ldd perl
        linux-vdso.so.1 (0x00007ffc8a09a000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f525fd82000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f525fd7d000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f525fc96000)
        libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f525fc5c000)
        libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007f525fc57000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f525fa2f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f525fda0000)

This makes the rules non-hermetic and prevents usage in clean environments like nixOS.
The issue also has been raised with the source of the perl binaries some time ago: skaji/relocatable-perl#13

Some background:

I want to compile openssl hermetically within bazel.

Incompatible flag --incompatible_disable_target_provider_fields will break rules_perl once Bazel 1.2.1 is released.

Please see the following CI builds for more information:

• [:ubuntu: 16.04 (OpenJDK 8)](:ubuntu: 16.04 (OpenJDK 8))
• [:darwin: (OpenJDK 8)](:darwin: (OpenJDK 8))

Questions? Please file an issue in https://github.com/bazelbuild/continuous-integration

Important: Please do NOT modify the issue title since that might break our tools.

Environment info:

uname -a
# Linux login2.longhorn.tacc.utexas.edu 4.14.0-115.10.1.el7a.ppc64le #1 SMP Wed Jun 26 09:32:17 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux

Problem:

build aborted: no matching toolchains found for types @rules_perl//:toolchain_type

With the exception of windows, all the other toolchains are currently downloaded from https://github.com/skaji/relocatable-perl . This is potentially an issue as now, even though the rules are authored by this org, this binaries aren't and therefore I have to implicitly trust https://github.com/skaji/relocatable-perl . It would be nice if those archives, like the window one, were mirrored in https://mirror.bazel.build/ .

Incompatible flag --incompatible_run_shell_command_string will break rules_perl once Bazel 1.2.1 is released.

Please see the following CI builds for more information:

• [:darwin: (OpenJDK 8)](:darwin: (OpenJDK 8))

Questions? Please file an issue in https://github.com/bazelbuild/continuous-integration

Important: Please do NOT modify the issue title since that might break our tools.

It would be very nice if a new release could be tagged now that support for bzlmod has been added. It can then be added to the Bazel Central Registry, which will make it easier for other modules to depend on rules_perl. If a release is tagged, I can do the work to submit a pull request to BCR to have it added, though longer term it would be good for this to be automated.

I'm trying to just import rules_perl via http_archive, but the repository isn't setup to make that trivial:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_perl",
    urls = ["https://github.com/bazelbuild/rules_perl/archive/9003b044f8555d7b7f4da051aa0f4b5616e65465.tar.gz"],
    sha256 = "0d1eabe253ca8f1a9270979869ef5f5b804727c0e4aaa85f14abf7d1827ca3cb",
)

load("@rules_perl//perl:deps.bzl", "perl_register_toolchains", "perl_rules_dependencies")

perl_rules_dependencies()
perl_register_toolchains()

ERROR: error loading package '': Every .bzl file must have a corresponding package, but '@rules_perl//perl:deps.bzl' does not have one. Please create a BUILD file in the same or any parent directory. Note that this BUILD file does not need to do anything except exist.

If I add a trivial build file, build_file_content = "# noop ",

ERROR: error loading package '': Label '@rules_perl//perl:deps.bzl' is invalid because 'perl' is not a package; perhaps you meant to put the colon here: '@rules_perl//:perl/deps.bzl'?

If I change the import: load("@rules_perl//:perl/deps.bzl", "perl_register_toolchains", "perl_rules_dependencies")

ERROR: error loading package '': cannot load '@rules_perl//:perl/deps.bzl': no such file

Currently darwin-aarch64's toolchain is not explicitly declared and as such it picks up the one for darwin-x86_64. This causes perl's toolchain not to work on m1/m2. A good example would be building openssl via rules_foreign_cc. Luckily the newest release of relocatable-perl seems to provide said toolchain.

Incompatible flag --incompatible_no_implicit_file_export will break rules_perl once Bazel 1.2.1 is released.

Please see the following CI builds for more information:

• [:ubuntu: 16.04 (OpenJDK 8)](:ubuntu: 16.04 (OpenJDK 8))
• [:darwin: (OpenJDK 8)](:darwin: (OpenJDK 8))

Questions? Please file an issue in https://github.com/bazelbuild/continuous-integration

Important: Please do NOT modify the issue title since that might break our tools.

This is the tracking issue for what's described in the README:

This way folks can subscribe to something in case this functionality gets added.

I expect those BUILD rules to work:

perl_xs(
    name = "CacheMemcachedFastXS",
    srcs = ["Fast.xs"],
    defines = [
        "VERSION=\"0.26\"",
        "XS_VERSION=\"0.26\"",
    ],
    output_loc = "arch/auto/Cache/Memcached/Fast/Fast.so",
    textual_hdrs = ["ppport.h"],
    typemaps = ["typemap"],
    deps = [":libclient"],
)

cc_library(
    name = "libclient",
    srcs = [
        "src/array.c",
        "src/array.h",
        "src/client.c",
        "src/compute_crc32.c",
        "src/compute_crc32.h",
        "src/connect.c",
        "src/connect.h",
        "src/dispatch_key.c",
        "src/dispatch_key.h",
        "src/parse_keyword.c",
        "src/parse_keyword.h",
        "src/socket_posix.c",
        "src/socket_posix.h",
    ],
    hdrs = [
        "src/client.h",
    ],
    copts = ["-I."],
    defines = [
        "HAVE_POLL_H",
    ],
    includes = [
        ".",
        "src",
    ],
)

(The complete example is available here: cache_memcached_fast.BUILD)

However, the :libclient dependency is completely ignored, and produced Fast.so is failing at runtime due to missing symbols. I'm not sure why Bazel doesn't pick up this dependency, since CcInfos are merged in perl.bzl.

I've spent more time looking into this that I'm willing to admit, but I couldn't make it work. Any ideas, @skeletonkey?

I can workaround this issue by pulling :libclient sources into :CacheMemcachedFastXS using #23 (see: diff), but that's literally a workaround and not a proper solution.

Example:

BUILD:

load("@rules_perl//perl:perl.bzl", "perl_binary")
load("@rules_python//python:defs.bzl", "py_binary")

perl_binary(
    name = "perl",
    srcs = ["perl.pl"],
)

py_binary(
    name = "python",
    srcs = ["python.py"],
    data = [":perl"],
    deps = ["@rules_python//python/runfiles"],
)

WORKSPACE:

workspace(name = "perlrulesbug")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_python",
    sha256 = "954aa89b491be4a083304a2cb838019c8b8c3720a7abb9c4cb81ac7a24230cea",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.4.0/rules_python-0.4.0.tar.gz",
)


http_archive(
    name = "rules_perl",
    sha256 = "9661de92edd38cc878010a4e42088f5ffda3f1e8b7d1124a5cea943e2fb03c0f",
    strip_prefix = "rules_perl-03de09c1dff31920d7bedda0d519e178dd1f1448",
    url = "https://github.com/bazelbuild/rules_perl/archive/03de09c1dff31920d7bedda0d519e178dd1f1448.zip",
)

load("@rules_perl//perl:deps.bzl", "perl_register_toolchains", "perl_rules_dependencies")

perl_rules_dependencies()

perl_register_toolchains()

perl.pl:

use strict;
use warnings;

print("Hello from Perl!\n");

python.py:

import subprocess
from pprint import pprint
from rules_python.python.runfiles import runfiles

my_runfiles = runfiles.Create()
perl_script = my_runfiles.Rlocation("perlrulesbug/perl")

subprocess.call([perl_script])

Test:

$ bazel run //:python
INFO: Analyzed target //:python (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //:python up-to-date:
  bazel-bin/python
INFO: Elapsed time: 0.051s, Critical Path: 0.00s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Traceback (most recent call last):
  File "/home/user1/.cache/bazel/_bazel_user1/ed6bf9cf7ba68480a3230f4c79ec9c4c/execroot/perlrulesbug/bazel-out/k8-fastbuild/bin/python.runfiles/perlrulesbug/python.py", line 8, in <module>
    subprocess.call([perl_script])
  File "/usr/lib/python3.9/subprocess.py", line 349, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.9/subprocess.py", line 1696, in _execute_child
    and os.path.dirname(executable)
  File "/usr/lib/python3.9/posixpath.py", line 152, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Despite the data dependency, the runfiles library cannot find perlrulesbug/perl.

It does not appear in the file bazel-out/k8-fastbuild/bin/python.runfiles_manifest or anywhere in the directory bazel-out/k8-fastbuild/bin/python.runfiles. It's like the data dependency just does not work at all. Is perl_binary not communicating its files correctly via DefaultInfo?

Example repo: https://github.com/jesseschalken/perlrulesbug

Incompatible flag --incompatible_no_implicit_file_export will be enabled by default in a future Bazel release [1], thus breaking rules_perl.

The flag is documented here: bazelbuild/bazel#10225

Please check the following CI builds for build and test results:

• Ubuntu 16.04, OpenJDK 8
• macOS, OpenJDK 8

Never heard of incompatible flags before? We have documentation that explains everything.

If you don't want to receive any future issues for rules_perl or if you have any questions,
please file an issue in https://github.com/bazelbuild/continuous-integration

Important: Please do NOT modify the issue title since that might break our tools.

[1] The target release hasn't been determined yet. Our tool will update the issue title once the flag flip has been scheduled.

bzlmod requires stable download urls. This can only be achieved by generating a tarball from a tag or release on github otherwise the sha of the downloaded tarballs isn't guarenteed to be consistent over time.