Cross compilation fails on Rust nightly when using proc macro crates due to GCC not understanding `-fuse-ld=lld` about rust HOT 12 CLOSED

I mean that the new behavior violates of the spirit of the target tier policy, but not the very precise guarantees we document.

from rust.

I had expected us to have a gcc version check already, especially since our dist i686/x64 linux CI builders are based on CentOS 7, but it seems the original -Zgcc-ld=lld implementation doesn't have such a version check, and that CI had gcc 9.5 available because it actually builds it from source using the GCC 4.8.5 package available.

I'll look into adding the version check.

from rust.

Also reported as cross-rs/cross#1496 but it seems to me that it is a rust regression that rust tries to use a gcc option when the installed GCC version is too old to support said option.

from rust.

I agree that this is a regression. Our tier 1 targets support GLIBC 2.17+ which dates back to 2012 but -fuse-ld=lld in gcc was merged in 2018. While we do not explicitly declare support for any gcc or distro version, GLIBC 2.17+ is generally interpreted as RHEL/CentOS 7.

Unfortunately, the platforms that the toolchain can run on often constrains what platforms can be targeted. We currently do not provide a mechanism to target GLIBC versions older than the host.

So I do not think it is not a big stretch to argue that passing -fuse-ld=lld without a check for a compatible gcc breaches the target tier support that we declare.

from rust.

So I do not think it is not a big stretch to argue that passing

@saethlin The double negation is hurting my brain, what do you actually mean here?

from rust.

Hmm, it seems

• version detection was deliberately avoided in the implementation, not forgotten
• and that this issue of old GCCs was discussed in #97402 where consensus at the time (amongst participants, not a wide-reaching discussion to be clear) seemed to be "there was no need to support GCCs this old". Which also could explain why we don't have GCC 4 when running tests on CI.

Maybe we need to reassess both of these and discuss more broadly, and/or document more precisely what we intend to rely on in the target support documentation, and/or more easily offer feature detection for users where these cases are important.

The latter could be a workflow like the following, where users want to dynamically opt-out of using lld:

• until stabilization, check if using a nightly compiler -- and use -Z/-Zunstable-options flags that won't be needed in the future
• running rustc --print linker-features returns whether lld is supported (we'd need to add support for linker-features in rustc --print first of course). We could also check if rust-lld exists in the sysroot here, instead of only as a fallback during compilation like #125263.
• if that succeeds (handwaving details), then users wanting to opt-out could do so with -Z/-Clinker-features=-lld. Of course, here they could also add their own dynamic opt-in/opt-out checks, like a GCC version check.
• otherwise, the default linker would not be lld in the first place, and nothing would need to be done

That would allow for users supporting multiple versions of the compiler and multiple versions of GCC if they need to.

cc @petrochenkov what do you think?

from rust.

My use case is building in CI, both for testing and for distributable binaries. I don't run old distros myself (I run Arch, which is rolling release). I just rely on cross selecting a suitable docker image such that it can cross compile binaries that are portable to any relevant glibc based Linux distros. And it seems that works okay (even on nightly), until it tries to build a proc macro that should execute on the host in the docker container.

Now I would say that "building portable binaries for distribution" is an important use case for Rust. Any binary crate that is not targeted at rust developers might want to do this. I don't expect my users to have a rust toolchain (or any toolchain at all) installed. So asking them to use cargo install does not make much sense.

Because of the unfortunate situation on Linux with glibc backward compatibility, the only reasonable way to do this is to build on an old distro (since glibc makes sure the sure the binary is forwards compatible, but not the reverse).

I believe this is a use case that needs to work. I don't particularly care about how it is made to work. Maybe rust can just bypass cc and call lld directly for example.

from rust.

WG-prioritization assigning priority (Zulip discussion).

@rustbot label -I-prioritize +P-high

from rust.

On re-read it looks like in #97402 we largely assumed that LLD is enabled explicitly somehow. In that case it's quite reasonable to not support LLD on systems where GCC doesn't support it, because that's a quite small version window (between the appearance of functioning LLD and -fuse-ld=lld support in GCC).

But now LLD is enabled implicitly in rustc and that increases the unsupported version window significantly (now it's from around 2012 to around 2018).
So I agree that we need to fix this somehow.

from rust.

One possible solution is a hack similar to fallback for unsupported -no-pie or -static-pie -

from rust.

I'll look into doing that, it seems preferable to doing the version detection on the happy path that I was working on.

from rust.

Another possible solution is to keep a fifth copy of lld-wrapper with a name ld (or ld.exe) in addition to these four

Also self-contained lld-named-ld will conflict with self-contained real-ld on targets where both are shipped (like windows-gnu).
But this can also be solved, by moving them to separate directories.

Let's try the linker rerun hack first, I think.

from rust.