Accessing large static global behaves incorrectly about rust HOT 10 OPEN

This issue is caused by a known problem in LLVM, which is the use of unsigned instead of size_t in this group of functions: https://llvm.org/doxygen/group__LLVMCCoreValueConstantComposite.html. On x86_64, unsigned is 32 bits, giving us an overflow at a length 2^32 array. This explains the inclusion of a symbol but no data - the linker is "correctly" including a zero-sized array.

This was encountered by Zig at ziglang/zig#1424, and one of their folks created an updated version of LLVMConstArray which was accepted into LLVM 17. This was also acknowledged in rustc https://github.com/rust-lang/rust/blame/89b78304e82dc5114e3b2faa0fbec747a28a2b37/compiler/rustc_codegen_llvm/src/llvm/ffi.rs#L951.

Unfortunately this doesn't help because the problem call is actually to LLVMConstStringInContext, made here

The best fix would be to update all of these functions in LLVM (why didn't the first PR just update them all? :P), then eventually integrate them here.

In the meantime, we can add correct implementations of these functions to rustc_llvm/llvm-wrapper/RustWrapper.cpp. This is not a complete fix because introducing huge statics requires a larger memory model when linking (https://www.smcm.iqfr.csic.es/docs/intel/compiler_f/main_for/bldaps_for/lin/bldaps_spec_memmods.htm). After doing this fix, the given program won't link on my end for this reason. I don't know how easy it is to change the memory model that rustc invokes.

Alternatively we can just add overflow checks to all of the callsites and disallow 2^32-length constant arrays & strings. GCC & Clang also choke on const char VALS[1024l * 1024l * 1024l * 4l] = {0}; on my machine, fwiw.

If the goal is just to have a huge static block of zeros at runtime, you can also just make VALS mutable. This moves it into .bss, sidestepping the entire issue. Also, the reason the playground doesn't like this code is because the const evaluator must generate and store those 2^32 zeros in memory.

from rust.

In the playground trying to get the llvm ir or output assembly results in a crash. Maybe there is an overflow at the llvm ir level?

from rust.

Emitting asm/llvm works fine for me on CLI, I think it's just a playground issue

from rust.

@rustbot claim

from rust.

Thanks for the info. I actually initially ran into this problem while trying to make a large mutable array, but I used the immutable example in the report because it didn't require unsafe to access (I forgot this would cause it not to go in the BSS). The following example still breaks, even though it should be possible to put the data in the BSS (I think).

static mut VALS: [u8; 1024 * 1024 * 1024 * 4] = [0u8; 1024 * 1024 * 1024 * 4];

fn main() {
    unsafe {
        for i in 0..2048 {
            println!("{}", VALS[i]);
            if VALS[i] != 0 {
                println!("NON-ZERO: {}", VALS[i]);
                return
            }
        }
    }
}

from rust.

Oh that's my mistake - I had the accesses commented out and it optimized VALS out :P. It does work for C, though. You could use that fact if this is something you really wanted.

from rust.

WG-prioritization assigning priority (Zulip discussion).

@rustbot label -I-prioritize +P-critical

from rust.

Priority downgraded (Zulip discussion) as it's not a release blocker and in out there since some time.

@rustbot label -I-prioritize +P-high -P-critical

from rust.

This will be fixed by #122000

from rust.

I don't know how easy it is to change the memory model that rustc invokes.

It's -C code-model=large, and can be set using target triple rustflags in Cargo.toml. https://doc.rust-lang.org/cargo/reference/config.html?highlight=rustflags#targettriplerustflags

Also, please don't use static mut! Instead: static VALS: SyncUnsafeCell<[u8; 1024 * 1024 * 1024 * 4]> = ...;

from rust.