scaleweather / eccodes Goto Github PK

ecCodes 2.33.0 manual build with POSIX-threads enabled.

Minimal example:

use std::{ffi::CString, thread};

fn main() {
    let t1 = thread::spawn(|| thread1("T1"));
    let t2 = thread::spawn(|| thread1("T3"));

    t1.join().unwrap();
    t2.join().unwrap();
}

fn thread1(name: &str) {
    let keys = CString::new("shortName, typeOfLevel").unwrap();
    let grib_file = CString::new("./data/iceland-surface.grib").unwrap();

    let mut err = 0;
    let ci;
    unsafe {
        let ctx = eccodes_sys::codes_context_get_default();

        println!("{name}: index_new");
        ci = eccodes_sys::codes_index_new(ctx, keys.as_ptr(), &mut err);
        println!("{name}: index_new: {err:?}");

        println!("{name}: index_add_file");
        err = eccodes_sys::codes_index_add_file(ci, grib_file.as_ptr());
        println!("{name}: index_add_file: {err:?}");

        thread::sleep(std::time::Duration::from_secs(3));

        println!("{name}: index_delete");
        eccodes_sys::codes_index_delete(ci);
        println!("{name}: index deleted");
    }
}

With eventually fail like so:

 cargo run                                                                                                                                
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target/debug/ec-test`
T1: index_new
T3: index_new
T1: index_new: 0
T1: index_add_file
T3: index_new: 0
T3: index_add_file
T1: index_add_file: -11
[1]    112709 segmentation fault (core dumped)  cargo run

Describe the bug
After following documentation, command cargo build crashed with the following error:

error[E0432]: unresolved import `eccodes_sys::_IO_FILE`
  --> /Users/bastienlecorre/.cargo/registry/src/index.crates.io-6f17d22bba15001f/eccodes-0.6.9/src/intermediate_bindings.rs:17:27
   |
   |     CODES_TYPE_UNDEFINED, _IO_FILE,
   |                           ^^^^^^^^ no `_IO_FILE` in the root

To Reproduce
Steps to reproduce the behavior:

1. cargo add eccodes
2. cargo build

Expected behavior
The compile process should finish successfully.

Screenshots

Desktop

• OS: MacOS
• Version: Sonoma 14.0

Additional context
New to Rust, maybe an error on my part?

Is your feature request related to a problem? Please describe.
As GRIB-files are binary dumps of message-sections index-files are used to encode the precalculated binary location of messages matching a set of keys. Some GRIB-files contains thousands of messages and to facilitate fast reading of messages index-files are delivered with the files and use to query the binary location of a message and receive handles. Loading GRIB-files by reading an index-file is required for performance in certain situations.

Describe the solution you'd like
I would like safe rust bindings for reading from GRIB index-files.

Additional context
I would like to use this crate to eventually migrate from C++ for working with GRIB and this is a required feature. I wouldn't mind trying to implement this feature myself but I'm still inexperienced working with rust and starting off with unsafe rust seems a bit daunting.

Index-files are supported by the eccodes C-api. It's also possible to create an index file quite simply with eccodes CLI-tools to get something to experiment with.

I believe this is the C-interface for reading from an existing index
https://github.com/onyb/eccodes/blob/master/src/eccodes.h#L242

codes_index* codes_index_read(codes_context* c,const char* filename,int *err);

The workflow is to index a file using a set of predetermined keys, such as ["shortName","typeOfLevel","level","stepType"]. You then read the index-file, selecting a value for each of the keys the index was created with, e.g. selecting

codes_index_select_string(index,"shortName","t");
codes_index_select_string(index,"typeOfLevel","heightAboveGround");
codes_index_select_string(index,"level",2);
codes_index_select_string(index,"stepType","instantaneous");

and then retreiving the grib-handle from the index using codes_handle_new_from_index