lbcb-sci / herro Goto Github PK

Dear,
I encountered some issues in the final step of the herro pipeline.
In the last command (herro inference), the following error emerges:

[00:00:01] Parsed 2313 reads.                                                                                                                                                                              
[00:00:05] Processing 1/? batch ⢀
[>---------------------------------------] 4/698                                                                                                                                                           
thread '<unnamed>' panicked at src/inference.rs:172:64:
called `Result::unwrap()` on an `Err` value: Torch("Expected at most 4 argument(s) for operator 'forward', but received 5 argument(s). Declaration: forward(__torch__.model.PositionClassifier self, Tensor bases, Tensor qualities, Tensor[] target_positions) -> Tensor\nException raised from checkAndNormalizeInputs at /libtorch/include/ATen/core/function_schema_inl.h:383 (most recent call first):\nframe #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) + 0x6b (0x7fc7aa65a6bb in /libs/libtorch/lib/libc10.so)\nframe #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) + 0xbf (0x7fc7aa6555ef in /libs/libtorch/lib/libc10.so)\nframe #2: void c10::FunctionSchema::checkAndNormalizeInputs<c10::Type>(std::vector<c10::IValue, std::allocator<c10::IValue> >&, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, c10::IValue, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, c10::IValue> > > const&) const + 0x45b (0x55f489b0f2eb in herro)\nframe #3: torch::jit::Method::operator()(std::vector<c10::IValue, std::allocator<c10::IValue> >, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, c10::IValue, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, c10::IValue> > > const&) const + 0x173 (0x7fc7af01b663 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #4: <unknown function> + 0x1ea9cd (0x55f489b0f9cd in herro)\nframe #5: <unknown function> + 0xaadaa (0x55f4899cfdaa in herro)\nframe #6: <unknown function> + 0xa9133 (0x55f4899ce133 in herro)\nframe #7: <unknown function> + 0xd2a11 (0x55f4899f7a11 in herro)\nframe #8: <unknown function> + 0xd9d0c (0x55f4899fed0c in herro)\nframe #9: <unknown function> + 0xf4a96 (0x55f489a19a96 in herro)\nframe #10: <unknown function> + 0x145375 (0x55f489a6a375 in herro)\nframe #11: <unknown function> + 0x94ac3 (0x7fc7aa46bac3 in /lib/x86_64-linux-gnu/libc.so.6)\nframe #12: <unknown function> + 0x126850 (0x7fc7aa4fd850 in /lib/x86_64-linux-gnu/libc.so.6)\n")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted (core dumped)

I was wondering if anybody can help me.

Best
Mario

Hi,

Thank you for your interesting tool.

I got a following error when I executed herro inference in Singurarity.

$ singurarity run --nv herro.sif inference <args>
[00:01:02] Parsed 393084 reads.                                                                                                                                  
[00:00:16] Processing 1/? batch ⠠
[>---------------------------------------] 2/75403                                                                                                               
Could not load library libcudnn_cnn_infer.so.8. Error: libcuda.so: cannot open shared object file: No such file or directory
Aborted

So, I entered into the bash console first, configurered the LD_LIBRARY_PATH environmental variable inside, and executed herro inference as below.

$ singurarity exec --nv herro.sif bash
Singularity> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.7/compat/
Singularity> herro inference <args>

Then, it worked.

Best

I'm getting following sequence error for batch.py script

File "./batch.py", line 43
if (idx := rids.get(tname, None)) is not None:
^
SyntaxError: invalid syntax

Hi,

I'm trying to run Herro using an A100 card. Not clear why it is running out of memory. I was running with -b 128 so I've dropped that down.

I'm running with singularity. Any suggestions?

Thanks

[00:01:26] Processing 1/? batch _
[>---------------------------------------] 93/90774 [W manager.cpp:340] Warning: FALLBACK path has been taken inside: runCudaFusionGroup. This is an indication that codegen Failed for some reason.
To debug try disable codegen fallback path via setting the env variable export PYTORCH_NVFUSER_DISABLE=fallback
(function runCudaFusionGroup)
thread '' panicked at src/inference.rs:172:64:
called Result::unwrap() on an Err value: Torch("The following operation failed in the TorchScript interpreter.\nTraceback of TorchScript (most recent call last):\nRuntimeError: The following operation failed in the TorchScript interpreter.\nTraceback of TorchScript, serialized code (most recent call last):\n File "code/torch/model.py", line 36, in fallback_cuda_fuser\n x0 = torch.permute(x, [0, 3, 1, 2])\n qn = self.qn\n sliced_sequences_concatenated = (qn).forward(x0, target_positions, lengths, )\n ~~~~~~~~~~~ <--- HERE\n fc2 = self.fc2\n _1 = (fc2).forward(sliced_sequences_concatenated, )\n File "code/torch/transformer.py", line 16, in forward\n _0 = torch.torch.nn.utils.rnn.pad_sequence\n context_read = self.context_read\n x0 = (context_read).forward(x, )\n ~~~~~~~~~~~~~~~~~~~~~ <--- HERE\n context_pos = self.context_pos\n x1 = (context_pos).forward(x0, )\n File "code/torch/torch/nn/modules/container.py", line 15, in forward\n _2 = getattr(self, "2")\n input0 = (_0).forward(input, )\n input1 = (_1).forward(input0, )\n ~~~~~~~~~~~ <--- HERE\n return (_2).forward(input1, )\n def len(self: torch.torch.nn.modules.container.Sequential) -> int:\n File "code/torch/torch/nn/modules/batchnorm.py", line 35, in forward\n weight = self.weight\n bias = self.bias\n _3 = _0(input, running_mean, running_var, weight, bias, bn_training, 0.10000000000000001, 1.0000000000000001e-05, )\n ~~ <--- HERE\n return _3\n def _check_input_dim(self: torch.torch.nn.modules.batchnorm.BatchNorm2d,\n File "code/torch/torch/nn/functional.py", line 52, in batch_norm\n else:\n pass\n _6 = torch.batch_norm(input, weight, bias, running_mean, running_var, training, momentum, eps, True)\n ~~~~~~~~~~~~~~~~ <--- HERE\n return _6\ndef relu(input: Tensor,\n\nTraceback of TorchScript, original code (most recent call last):\n File "/raid/scratch/stanojevicd/projects/haec-BigBird/model.py", line 157, in fallback_cuda_fuser\n sliced_sequences_concatenated = torch.cat(encoded)'''\n x = x.permute((0, 3, 1, 2))\n sliced_sequences_concatenated = self.qn(x, target_positions, lengths)\n ~~~~~~~ <--- HERE\n \n # list of tensors of shape (selected_token_number, 1) -> (selected_token_number)\n File "/raid/scratch/stanojevicd/projects/haec-BigBird/transformer.py", line 36, in forward\n def forward(self, x: Tensor, target_positions: List[Tensor],\n lengths: Tensor) -> Tensor:\n x = self.context_read(x) # [B, I, L, R] -> [B, 128, L, R]\n ~~~~~~~~~~~~~~~~~ <--- HERE\n x = self.context_pos(x) # [B, 128, L, R] -> [B, 256, L, 1]\n x = x.squeeze(-1).transpose(1, 2) # [B, L, 256]\n File "/home/stanojevicd/miniforge3/envs/haec/lib/python3.11/site-packages/torch/nn/modules/container.py", line 215, in forward\n def forward(self, input):\n for module in self:\n input = module(input)\n ~~~~~~ <--- HERE\n return input\n File "/home/stanojevicd/miniforge3/envs/haec/lib/python3.11/site-packages/torch/nn/modules/batchnorm.py", line 171, in forward\n used for normalization (i.e. in eval mode when buffers are not None).\n """\n return F.batch_norm(\n ~~~~~~~~~~~~ <--- HERE\n input,\n # If buffers are not to be tracked, ensure that they won't be updated\n File "/home/stanojevicd/miniforge3/envs/haec/lib/python3.11/site-packages/torch/nn/functional.py", line 2478, in batch_norm\n _verify_batch_size(input.size())\n\n return torch.batch_norm(\n ~~~~~~~~~~~~~~~~ <--- HERE\n input, weight, bias, running_mean, running_var, training, momentum, eps, torch.backends.cudnn.enabled\n )\nRuntimeError: CUDA out of memory. Tried to allocate 4.64 GiB (GPU 0; 9.50 GiB total capacity; 4.74 GiB already allocated; 1.93 GiB free; 7.06 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF\n\n")
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace
Aborted (core dumped)

Getting the following error:

❯ nice nice singularity run --nv herro.sif inference -m model_R9_v0.1.pt -b 64 --read-alns batched_alignments out.fastq.gz corrected
[00:44:38] Parsed 9283361 reads.
[00:03:06] Processing 1/? batch ⠠
[>---------------------------------------] 3/68175
[W graph_fuser.cpp:108] Warning: operator() profile_node %1243 : int[] = prim::profile_ivalue(%1241)
[00:04:51] Processing 1/? batch ⢀
[>---------------------------------------] 844/68175
thread '<unnamed>' panicked at /herro/src/haec_io.rs:144:9:
Out of bounds for 2-bit sequence decoding.
[00:04:51] Processing 1/? batch ⠐
[>---------------------------------------] 844/68175                                                                                      0:     0x5567db11c2b1 - std::backtrace_rs::backtrace::libunwind::trace::ha637c64ce894333a
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5
   1:     0x5567db11c2b1 - std::backtrace_rs::backtrace::trace_unsynchronized::h47f62dea28e0c88d
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x5567db11c2b1 - std::sys_common::backtrace::_print_fmt::h9eef0abe20ede486
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x5567db11c2b1 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hed7f999df88cc644
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x5567db067390 - core::fmt::rt::Argument::fmt::h1539a9308b8d058d
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/fmt/rt.rs:142:9
   5:     0x5567db067390 - core::fmt::write::h3a39390d8560d9c9
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/fmt/mod.rs:1120:17
   6:     0x5567db11a3ff - std::io::Write::write_fmt::h5fc9997dfe05f882
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/io/mod.rs:1762:15
   7:     0x5567db11c094 - std::sys_common::backtrace::_print::h894006fb5c6f3d45
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x5567db11c094 - std::sys_common::backtrace::print::h23a2d212c6fff936
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x5567db11d4a7 - std::panicking::default_hook::{{closure}}::h8a1d2ee00185001a
  10:     0x5567db11d205 - std::panicking::default_hook::h6038f2eba384e475
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:292:9
  11:     0x5567db11da40 - std::panicking::rust_panic_with_hook::h2b5517d590cab22e
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:779:13
  12:     0x5567db11d7b9 - std::panicking::begin_panic_handler::{{closure}}::h233112c06e0ef43e
13:     0x5567db11c776 - std::sys_common::backtrace::__rust_end_short_backtrace::h6e893f24d7ebbff8
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys_common/backtrace.rs:170:18
  14:     0x5567db11d572 - rust_begin_unwind
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/panicking.rs:645:5
  15:     0x5567dafff975 - core::panicking::panic_fmt::hbf0e066aabfa482c
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/core/src/panicking.rs:72:14
  16:     0x5567db094cd6 - herro::features::extract_features::{{closure}}::h3588956aabde7bf6
  17:     0x5567db0944a4 - core::slice::sort::insertion_sort_shift_left::h6a8cb7ba2acaadd2
  18:     0x5567db093da6 - core::slice::sort::merge_sort::ha0f0096f1486e1d5
  19:     0x5567db0ac7b8 - herro::features::extract_features::h00ea577d98be430b
  20:     0x5567db0b5a6d - std::sys_common::backtrace::__rust_begin_short_backtrace::hd2c9f1af20faea53
  21:     0x5567db0d0be6 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h44881a552781c61d
  22:     0x5567db121375 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hc7eafaff61e32df9
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/alloc/src/boxed.rs:2007:9
  23:     0x5567db121375 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h6ba4a5de48dd2304
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/alloc/src/boxed.rs:2007:9
  24:     0x5567db121375 - std::sys::unix::thread::Thread::new::thread_start::he469335aef763e45
                               at /rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/sys/unix/thread.rs:108:17
  25:     0x7f0a5d66bac3 - <unknown>
  26:     0x7f0a5d6fca04 - __clone
  27:                0x0 - <unknown>
Aborted (core dumped)

Hi,
I'm wondering if this method also would be applicable to a datasets of polyploid species. Any thoughts towards that direction?

Thanks!

Hello,
can you please describe how herro's error correction steps work, and how you ensure the preservation of the phase between two variant sites within the same read? I see that, though N50 stats increase in your assemblies, switch and hamming rate remain the same.
Other questions:
Will herro work also with R9 data? Is there a minimum ONT coverage that the software needs to do the error correction? Can I run herro without GPUs?
Thanks,

Dario

Hi,

1. What is the minimum GPU VRAM required for running HERRO?
2. How does HERRO perform on error correction of ONT duplex reads?

Best,
Bikram Panda

Hi,
I was wondering if it is possible to obtain summary stats on the number of heterozygous SNPs for HG002 sample using corrected reads and compare with PacBio HiFi or GIAB data against CHM13-T2T or GRCh38. This will indicate if there is overcorrection in favour of a single allele at a locus. I understand that you are preparing manuscript and this may be included in it. Awaiting results eagerly.
Cheers

Hello

I have used herro for several data sets of insect whole genome sequence, and obtained amazing results.
Herro + hifiasm generated more contiguous and high BUSCO score assemblies compared to flye while demanding less amount of memory.

• Question
  It seems that preprocess.sh omits reads shoter than 10,000 bp.
  Is it because it's nesessary, or is it no problem to include shoter reads when assembling relatively small genomes?

Thanks

Hey herro team,
I'd like to test herro but unfortunately I face issues with installation step 1, the conda environment. The build fails when trying to download Porechop via pip. Do you see anything being wrong about my setup or the way I am installing herro?

• Error message:

where each /path/to was manually censored from the log.

• Details:

In general, never before have I been asked for password authentication when installing a conda env. So, I inspected the herro-env.yml file and found this line git+https://github.com/dehui333/Porechop.git in the pip section. I haven't seen such an install instruction before but, assuming that this works in general, I tried to look up the URL. However, if I copy+paste the Porechop URL into my browser I get an Error 404: not found. Also, when using GitHub's global search function there is (as of today) no public repository Porechop by a user dehui333.

• System and setup:

NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"

conda 24.1.2

Hello,
I try to run the Error-correction step but it doesn't work I get the following error:

thread '' panicked at /herro/src/inference.rs:197:70:
Cannot load model.: Torch("Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx\nException raised from device_count_impl at ../c10/cuda/CUDAFunctions.cpp:44 (most recent call first):\nframe #0: c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits, std::allocator >) + 0x6b (0x2af52f05a6bb in /libs/libtorch/lib/libc10.so)\nframe #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, char const*) + 0xc9 (0x2af52f055769 in /libs/libtorch/lib/libc10.so)\nframe #2: c10::cuda::device_count_ensure_non_zero() + 0xd8 (0x2af52f64afe8 in /libs/libtorch/lib/libc10_cuda.so)\nframe #3: + 0x103931a (0x2af4d1e3931a in /libs/libtorch/lib/libtorch_cuda.so)\nframe #4: + 0x2c30f36 (0x2af4d3a30f36 in /libs/libtorch/lib/libtorch_cuda.so)\nframe #5: + 0x2c30ffb (0x2af4d3a30ffb in /libs/libtorch/lib/libtorch_cuda.so)\nframe #6: at::_ops::empty_strided::redispatch(c10::DispatchKeySet, c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::optionalc10::ScalarType, c10::optionalc10::Layout, c10::optionalc10::Device, c10::optional) + 0x1fb (0x2af518eb71fb in /libs/libtorch/lib/libtorch_cpu.so)\nframe #7: + 0x25ebc75 (0x2af5191ebc75 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #8: at::_ops::empty_strided::call(c10::ArrayRefc10::SymInt, c10::ArrayRefc10::SymInt, c10::optionalc10::ScalarType, c10::optionalc10::Layout, c10::optionalc10::Device, c10::optional) + 0x168 (0x2af518ef2328 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #9: + 0x1701f5f (0x2af518301f5f in /libs/libtorch/lib/libtorch_cpu.so)\nframe #10: at::native::_to_copy(at::Tensor const&, c10::optionalc10::ScalarType, c10::optionalc10::Layout, c10::optionalc10::Device, c10::optional, bool, c10::optionalc10::MemoryFormat) + 0x17e3 (0x2af5186a6cf3 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #11: + 0x27d3603 (0x2af5193d3603 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #12: at::_ops::_to_copy::redispatch(c10::DispatchKeySet, at::Tensor const&, c10::optionalc10::ScalarType, c10::optionalc10::Layout, c10::optionalc10::Device, c10::optional, bool, c10::optionalc10::MemoryFormat) + 0x103 (0x2af518b93c83 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #13: + 0x25f01c8 (0x2af5191f01c8 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #14: at::_ops::_to_copy::redispatch(c10::DispatchKeySet, at::Tensor const&, c10::optionalc10::ScalarType, c10::optionalc10::Layout, c10::optionalc10::Device, c10::optional, bool, c10::optionalc10::MemoryFormat) + 0x103 (0x2af518b93c83 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #15: + 0x3a66271 (0x2af51a666271 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #16: + 0x3a6681b (0x2af51a66681b in /libs/libtorch/lib/libtorch_cpu.so)\nframe #17: at::_ops::_to_copy::call(at::Tensor const&, c10::optionalc10::ScalarType, c10::optionalc10::Layout, c10::optionalc10::Device, c10::optional, bool, c10::optionalc10::MemoryFormat) + 0x201 (0x2af518c16651 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #18: at::native::to(at::Tensor const&, c10::Device, c10::ScalarType, bool, bool, c10::optionalc10::MemoryFormat) + 0xfd (0x2af5186a505d in /libs/libtorch/lib/libtorch_cpu.so)\nframe #19: + 0x29a5612 (0x2af5195a5612 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #20: at::_ops::to_device::call(at::Tensor const&, c10::Device, c10::ScalarType, bool, bool, c10::optionalc10::MemoryFormat) + 0x1c1 (0x2af518d95cd1 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #21: torch::jit::Unpickler::readInstruction() + 0x1719 (0x2af51b766789 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #22: torch::jit::Unpickler::run() + 0xa8 (0x2af51b767988 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #23: torch::jit::Unpickler::parse_ivalue() + 0x2e (0x2af51b76953e in /libs/libtorch/lib/libtorch_cpu.so)\nframe #24: torch::jit::readArchiveAndTensors(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optional<std::function<c10::StrongTypePtr (c10::QualifiedName const&)> >, c10::optional<std::function<c10::intrusive_ptr<c10::ivalue::Object, c10::detail::intrusive_target_default_null_typec10::ivalue::Object > (c10::StrongTypePtr, c10::IValue)> >, c10::optionalc10::Device, caffe2::serialize::PyTorchStreamReader&, c10::Type::SingletonOrSharedTypePtrc10::Type (*)(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&), std::shared_ptrtorch::jit::DeserializationStorageContext) + 0x529 (0x2af51b7241a9 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #25: + 0x4b08c4b (0x2af51b708c4b in /libs/libtorch/lib/libtorch_cpu.so)\nframe #26: + 0x4b0b04b (0x2af51b70b04b in /libs/libtorch/lib/libtorch_cpu.so)\nframe #27: torch::jit::import_ir_module(std::shared_ptrtorch::jit::CompilationUnit, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optionalc10::Device, std::unordered_map<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits, std::allocator > const, std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > >&, bool, bool) + 0x3a2 (0x2af51b70f6c2 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #28: torch::jit::import_ir_module(std::shared_ptrtorch::jit::CompilationUnit, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optionalc10::Device, bool) + 0x92 (0x2af51b70fa42 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #29: torch::jit::load(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, c10::optionalc10::Device, bool) + 0xd1 (0x2af51b70fb71 in /libs/libtorch/lib/libtorch_cpu.so)\nframe #30: + 0x1de74e (0x55867173674e in /usr/bin/herro)\nframe #31: + 0xf3e9c (0x55867164be9c in /usr/bin/herro)\nframe #32: + 0xd2758 (0x55867162a758 in /usr/bin/herro)\nframe #33: + 0xd9d0c (0x558671631d0c in /usr/bin/herro)\nframe #34: + 0xf4a96 (0x55867164ca96 in /usr/bin/herro)\nframe #35: + 0x145375 (0x55867169d375 in /usr/bin/herro)\nframe #36: + 0x94ac3 (0x2af52f337ac3 in /lib/x86_64-linux-gnu/libc.so.6)\nframe #37: clone + 0x44 (0x2af52f3c8a04 in /lib/x86_64-linux-gnu/libc.so.6)\n")

I ran the following command: singularity exec --bind $DIR $DIR/bin/herro_v0.1.sif herro inference --read-alns batch_aln -m herro_model_v0.1.pt -b 64 herro_split.fastq.gz herro_cor.fasta

Do you know what is the origin of the error?
Thanks.

I tried this command

apptainer run --nv \                                                                 
    --bind "/scratch":"/output" \
    --bind "/lizardfs/guarracino/ratty/sequencing-data/HXB10/ONT":"/input" \
    --bind "/lizardfs/guarracino/git/herro/resources":"/models" \
    herro.sif inference -d 0,1 -t 8 -b 32 -m /models/model_v0.1.pt /input/HXB10.fq.gz /output/HXB10.herro.fq.gz

I got this error:

INFO:    underlay of /etc/localtime required more than 50 (77) bind mounts
INFO:    underlay of /usr/bin/nvidia-debugdump required more than 50 (308) bind mounts
[00:03:31] Parsed 430454 reads.                                                                                                                                          
[00:52:46] Processing 1/? batch ⠄                                                                                                                                        
[00:52:46] Processing 1/? batch ⡀                                                                                                                                        
[00:55:02] Processing 1/? batch ⠁
[>---------------------------------------] 2/94965                                                                                                                       [W graph_fuser.cpp:108] Warning: operator() profile_node %1243 : int[] = prim::profile_ivalue(%1241)
[00:55:12] Processing 1/? batch ⠂
[>---------------------------------------] 19/94965                                                                                                                      thread '<unnamed>' panicked at src/inference.rs:172:64:
called `Result::unwrap()` on an `Err` value: Torch("The following operation failed in the TorchScript interpreter.\nTraceback of TorchScript, serialized code (most recent call last):\n  File \"code/__torch__/transformer.py\", line 26, in forward\n      _2 = torch.select(x2, 0, i)\n      _3 = annotate(List[Optional[Tensor]], [tp])\n      _4 = torch.append(batch, torch.index(_2, _3))\n                               ~~~~~~~~~~~ <--- HERE\n    x3 = _0(batch, True, 0., )\n    _5 = __torch__.transformer.create_mask(lengths, )\n\nTraceback of TorchScript, original code (most recent call last):\n  File \"/raid/scratch/stanojevicd/projects/haec-BigBird/transformer.py\", line 40, in forward\n        x = x.squeeze(-1).transpose(1, 2)  # [B, L, 256]\n    \n        batch = [x[i, tp] for i, tp in enumerate(target_positions)]\n                 ~~~~~~~~ <--- HERE\n        x = nn.utils.rnn.pad_sequence(batch, True)\n        mask = create_mask(lengths).to(device=x.device)\nRuntimeError: CUDA error: an illegal memory access was encountered\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1.\nCompile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.\n\n")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted (core dumped)

Can you help me?

I am doing 16S rna amplicon based ONT sequencing.

My target region is only of 1kb to 2kb in length.

Per sample reads is around 50K.

Will HEERO give me good error filtered estimates ?

Hi,
I would like to try Herro, as it looks very efficient. However, as many of us, I don't work on aws (and don't have an account). Is it possible to provide an other way to get access to the models and the image?

For the models, I would recommend using Zenodo (https://zenodo.org/). It is free and datasets can be up to 50 Gb, and your dataset will get a DOI and a direct download link. For the container image, I would recommend to store it in DockerHub (or the new github system), it will make it accessible and compatible with docker, singularity and apptainer users, and it will provide a version history.
Thanks for your program,
Best.

Hello,

I just thought I'd request a feature - would you consider implementing some kind of checkpointing for the minimap2 alignments prior to error correction (ie the "create_batched_alignments.sh" step)? I suspect this can be the most computationally expensive step of this pipeline for large datasets. I just had a pretty long job killed by our cluster before it could finish, and it's a bit devastating to have to relaunch from the beginning.

Thanks,
Chris