Giter VIP home page Giter VIP logo

Comments (3)

Weshnaw avatar Weshnaw commented on July 4, 2024 1

I debugged this a bit myself; and figured out why the whole thread crashes when the requests are canceled:

for (output, mut result_tx) in batch_output.into_iter().zip(batch_txs) {
    result_tx.send(output).expect("Channel from calling thread disconnected");
}

https://github.com/epwalsh/batched-fn/blob/main/src/lib.rs#L347-L349

this expect crashes the thread because gRPC drops the calling thread if the request is canceled; this might just be my own opinion but I think changing this to

for (output, mut result_tx) in batch_output.into_iter().zip(batch_txs) {
    result_tx.send(output).ok();
}

should be fine for most cases. What are we losing by not crashing with the calling thread disconnects?

from batched-fn.

epwalsh avatar epwalsh commented on July 4, 2024

Hey @Weshnaw, does indeed look like the same issue. Could run with RUST_BACKTRACE=1 and send me the output?

from batched-fn.

Weshnaw avatar Weshnaw commented on July 4, 2024 
2024-03-05T20:18:40.725817Z  INFO example: Initializing Tonic...
2024-03-05T20:18:40.726117Z  INFO tonic_startup: example: address: [::1]:50051
2024-03-05T20:18:40.726217Z  INFO tonic_startup: example: Initializing Tonic Reflection...
2024-03-05T20:18:40.727112Z  INFO example: Tonic Initialized...
2024-03-05T20:18:46.316296Z  INFO encode_sentence: example: EncodeSentence request received...
2024-03-05T20:18:46.316826Z  INFO batch_context: example: Initializing Model...
2024-03-05T20:18:46.370480Z  INFO batch_context:model_load: example: Cuda: true
2024-03-05T20:18:46.370635Z  INFO batch_context:model_load: example: Model: resources/instructor-xl
2024-03-05T20:18:48.616130Z  INFO encode_sentence: example: EncodeSentence request received...
thread '<unnamed>' panicked at example\src\main.rs:77:24:
Channel from calling thread disconnected: "SendError(..)"
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\panicking.rs:645
   1: core::panicking::panic_fmt
             at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\core\src\panicking.rs:72
   2: core::result::unwrap_failed
             at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\core\src\result.rs:1649
   3: enum2$<core::result::Result<tuple$<>,flume::SendError<alloc::vec::Vec<f32,alloc::alloc::Global> > > >::expect<tuple$<>,flume::SendError<alloc::vec::Vec<f32,alloc::alloc::Global> > >
             at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\core\src\result.rs:1030
   4: example::encode_setence::async_fn$0::BATCHED_FN::closure$0::closure$0
             at C:\Users\bfall\.cargo\registry\src\index.crates.io-6f17d22bba15001f\batched-fn-0.2.4\src\lib.rs:353
   5: core::hint::black_box
             at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\core\src\hint.rs:334
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
2024-03-05T20:18:54.889290Z  INFO encode_sentence: example: EncodeSentence request received...
2024-03-05T20:18:54.889483Z  WARN encode_sentence: example: Disconnected

here's also with the full BACKTRACE:

RUST_BACKTRACE=full 
2024-03-05T20:20:47.996248Z  INFO example: Initializing Tonic...
2024-03-05T20:20:47.996537Z  INFO tonic_startup: example: address: [::1]:50051
2024-03-05T20:20:47.996647Z  INFO tonic_startup: example: Initializing Tonic Reflection...
2024-03-05T20:20:47.997533Z  INFO example: Tonic Initialized...
2024-03-05T20:21:13.492921Z  INFO encode_sentence: example: EncodeSentence request received...
2024-03-05T20:21:13.493519Z  INFO batch_context: example: Initializing Model...
2024-03-05T20:21:13.515958Z  INFO batch_context:model_load: example: Cuda: true
2024-03-05T20:21:13.516112Z  INFO batch_context:model_load: example: Model: resources/instructor-xl
thread '<unnamed>' panicked at example\src\main.rs:77:24:
Channel from calling thread disconnected: "SendError(..)"
stack backtrace:
   0:     0x7ff6ab916712 - std::sys_common::backtrace::_print::impl$0::fmt
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\sys_common\backtrace.rs:44
   1:     0x7ff6ab92ff3d - core::fmt::rt::Argument::fmt
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\core\src\fmt\rt.rs:142
   2:     0x7ff6ab92ff3d - core::fmt::write
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\core\src\fmt\mod.rs:1120
   3:     0x7ff6ab912e11 - std::io::Write::write_fmt<std::sys::windows::stdio::Stderr>
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\io\mod.rs:1810
   4:     0x7ff6ab91653a - std::sys_common::backtrace::_print
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\sys_common\backtrace.rs:47
   5:     0x7ff6ab91653a - std::sys_common::backtrace::print
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\sys_common\backtrace.rs:34
   6:     0x7ff6ab918789 - std::panicking::default_hook::closure$1
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\panicking.rs:272
   7:     0x7ff6ab918445 - std::panicking::default_hook
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\panicking.rs:292
   8:     0x7ff6ab918cb4 - std::panicking::rust_panic_with_hook
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\panicking.rs:779
   9:     0x7ff6ab918b89 - std::panicking::begin_panic_handler::closure$0
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\panicking.rs:657
  10:     0x7ff6ab916db9 - std::sys_common::backtrace::__rust_end_short_backtrace<std::panicking::begin_panic_handler::closure_env$0,never$>
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\sys_common\backtrace.rs:171
  11:     0x7ff6ab918852 - std::panicking::begin_panic_handler
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\panicking.rs:645
  12:     0x7ff6abb6eb07 - core::panicking::panic_fmt
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\core\src\panicking.rs:72
  13:     0x7ff6abb6f143 - core::result::unwrap_failed
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\core\src\result.rs:1649
  14:     0x7ff6aae7ba75 - enum2$<core::result::Result<tuple$<>,flume::SendError<alloc::vec::Vec<f32,alloc::alloc::Global> > > >::expect<tuple$<>,flume::SendError<alloc::vec::Vec<f32,alloc::alloc::Global> > >
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\core\src\result.rs:1030
  15:     0x7ff6aae16586 - example::encode_setence::async_fn$0::BATCHED_FN::closure$0::closure$0
                               at C:\Users\bfall\.cargo\registry\src\index.crates.io-6f17d22bba15001f\batched-fn-0.2.4\src\lib.rs:353
  16:     0x7ff6aae01209 - core::hint::black_box
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\core\src\hint.rs:334
  17:     0x7ff6aae01209 - std::sys_common::backtrace::__rust_begin_short_backtrace<example::encode_setence::async_fn$0::BATCHED_FN::closure$0::closure_env$0,tuple$<> >
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\std\src\sys_common\backtrace.rs:155
  18:     0x7ff6aaeda781 - std::thread::impl$0::spawn_unchecked_::closure$1::closure$0<example::encode_setence::async_fn$0::BATCHED_FN::closure$0::closure_env$0,tuple$<> >
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\std\src\thread\mod.rs:529
  19:     0x7ff6aae52881 - core::panic::unwind_safe::impl$23::call_once<tuple$<>,std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<example::encode_setence::async_fn$0::BATCHED_FN::closure$0::closure_env$0,tuple$<> > >
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\core\src\panic\unwind_safe.rs:272
  20:     0x7ff6aaebacf5 - std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<example::encode_setence::async_fn$0::BATCHED_FN::closure$0::closure_env$0,tuple$<> > >,tuple$<> >
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\std\src\panicking.rs:552
  21:     0x7ff6aaec6333 - std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<rayon_core::join::join_context::call_a::closure_env$0<alloc::collections::linked_list::LinkedList<alloc::vec::Vec<rust_tokenizers::tokenizer::base_tokenizer::TokenizedInput,alloc::al
  22:     0x7ff6aaeb2384 - std::panicking::try<tuple$<>,core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$0::spawn_unchecked_::closure$1::closure_env$0<example::encode_setence::async_fn$0::BATCHED_FN::closure$0::closure_env$0,tuple$<> > > >
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\std\src\panicking.rs:516
  23:     0x7ff6aaeda5a1 - std::panic::catch_unwind
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\std\src\panic.rs:142
  24:     0x7ff6aaeda5a1 - std::thread::impl$0::spawn_unchecked_::closure$1<example::encode_setence::async_fn$0::BATCHED_FN::closure$0::closure_env$0,tuple$<> >
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\std\src\thread\mod.rs:528
  25:     0x7ff6aae97b8e - core::ops::function::FnOnce::call_once<std::thread::impl$0::spawn_unchecked_::closure_env$1<example::encode_setence::async_fn$0::BATCHED_FN::closure$0::closure_env$0,tuple$<> >,tuple$<> >
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1\library\core\src\ops\function.rs:250
  26:     0x7ff6ab91c9dc - std::sys::windows::thread::impl$0::new::thread_start
                               at /rustc/89e2160c4ca5808657ed55392620ed1dbbce78d1/library\std\src\sys\windows\thread.rs:58
  27:     0x7fffbaca257d - BaseThreadInitThunk
  28:     0x7fffbc8caa58 - RtlUserThreadStart
2024-03-05T20:21:21.515465Z  INFO encode_sentence: example: EncodeSentence request received...
2024-03-05T20:21:21.515662Z  WARN encode_sentence: example: Disconnected

from batched-fn.

Related Issues (7)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.