marbl / harvest-tools Goto Github PK

Rerooting code assumes a binary root. When the root has more than two children, rerooting operations have incorrect results.

Hi

I am trying to do an alignment of 190 Mycobacterium tuberculosis genome sequences using parsnp. It seems to work fine and creates an .xmfa file. However it doesn't recruit all the genome fasta files into the alignment - only 95 of the total 190. I'm sure why because all of the files are in fasta format, and it doesn't seem to be related to the file name.

Any advice would be greatly appreciated.

Thanks
Tasha

Dear Harvest developers,

I am trying to use your software to convert an xmfa file into a multi-fasta, using
harvesttools -x ${samplename}.xmfa -M ${samplename}.fasta
But it throws an error:
libc++abi.dylib: terminating with uncaught exception of type std::length_error: basic_string
Abort trap: 6

My best guess is that there must be something wrong with the input-file specification. However, it is possible to load the xmfa into the mauve viewer without any problems; and also when eyeballing the file itself nothing seems wrong.
I am working on MacOS 10.13.6 with 16 Gb of memory.

Do you know what might be causing this error, how I can diagnose it further, or how I could resolve it? Any help would be greatly appreciated!

I am upgrading the brew package to 1.2 and get this problem:

./configure --with-capnp=/bio/linuxbrew/opt/capnp --with-protobuf=/bio/linuxbrew/opt/protobuf
configure: WARNING: unrecognized options: --with-capnp
checking for protoc... yes
checking for capnp... yes
checking whether the C++ compiler works... yes
<snip>
config.status: creating Makefile
configure: WARNING: unrecognized options: --with-capnp

However, I think it has created the Makefile just fine, and it compiles.

It is currently possible to load files without writing, which is counterintuitive.

Possible improvements:

• provide a warning when imported data would be lost
• automatically write to the input Gingr file if no output was specified

I couldn't find in the tutorial.
it's not clear how i would know ahead of time which clades correspond to which track ids if that makes sense.

Thanks!

There should be an output format that includes gene and amino acid information for variants based on reference position and annotations, either in VCF out or a new format.

Currently, there is no easy way to extract clade defining SNPs. Add a function to harvest-tools to enable extraction of these SNPs for tie-in with Gingr and to also improve command-line access to this info.

Remove this check and use accession #s instead

I'm trying to add HarvestTools to EasyBuild, for use on HPC clusters.

Having included the fix to use C++17 which allows the configure step to complete, I see this error during build:

g++ -c -O2 -ftree-vectorize -march=native -fno-math-errno -std=c++17 -Isrc -I/apps/eb/el8/upstream/software/protobuf/23.0-GCCcore-12.2.0/include -I/apps/eb/el8/upstream/software/CapnProto/0.10.3-GCCcore-12.2.0/include -include src/harvest/memcpyLink.h -Wl,--wrap=memcpy -I/apps/eb/el8/upstream/software/CapnProto/0.10.3-GCCcore-12.2.0/include -I/apps/eb/el8/upstream/software/protobuf/23.0-GCCcore-12.2.0/include -I/apps/eb/el8/upstream/software/zlib/1.2.12-GCCcore-12.2.0/include -o src/harvest/pb/harvest.pb.o src/harvest/pb/harvest.pb.cc
src/harvest/HarvestIO.cpp: In member function bool HarvestIO::loadHarvestProtocolBuffer(const char*):
src/harvest/HarvestIO.cpp:211:39: error: no matching function for call to google::protobuf::io::CodedInputStream::SetTotalBytesLimit(int, int)
  211 |         coded_input.SetTotalBytesLimit(INT_MAX, INT_MAX);
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
In file included from src/harvest/pb/harvest.pb.h:24,
                 from src/harvest/HarvestIO.h:10,
                 from src/harvest/HarvestIO.cpp:7:
/apps/eb/el8/upstream/software/protobuf/23.0-GCCcore-12.2.0/include/google/protobuf/io/coded_stream.h:390:8: note: candidate: void google::protobuf::io::CodedInputStream::SetTotalBytesLimit(int)
  390 |   void SetTotalBytesLimit(int total_bytes_limit);
      |        ^~~~~~~~~~~~~~~~~~
/apps/eb/el8/upstream/software/protobuf/23.0-GCCcore-12.2.0/include/google/protobuf/io/coded_stream.h:390:8: note:   candidate expects 1 argument, 2 provided
make: *** [Makefile:38: src/harvest/HarvestIO.o] Error 1
make: *** Waiting for unfinished jobs....
 (at easybuild/tools/run.py:682 in parse_cmd_output)

I'm using protobuf 23.0 and CapnProto 0.10.3.

There seems to be discussion of a similar problem here:
onnx/onnx#2678

The HarvestIO interface should be documented for third party development, ideally with an automated tool such as Doxygen.

Hi,
is it possible to convert the xmfa file from Parsnp (1.7.4) to maf file?
I tried harvest-tools, but it seems do not have this function.
I also tried xmfa2maf (from Mauve https://github.com/Wyss/mauve-py/blob/master/mauve/src/mauveAligner/src/xmfa2maf.cpp), which failed to recognize the character "=" in between the alignment blocks.

When trying to run parsnp I get the following Error

Traceback (most recent call last):
File "", line 624, in
NameError: name 'nameok' is not defined

What might be the reason for that?
thx

Hi,
I received a patch to the Debian package of harvest-tools which enables parallel builds. Please feel free to apply this for your next release.
Kind regards, Andreas.

Hello,
the Debian Med team is maintaining harvest-tools for official Debian. The recently released Debian 10 was the last Debian release featuring Python2 since this programming language is EOL. If you are interested that we continue to maintain harvest-tools in official Debian (and that users of other modern distributions will have no problems to install harvest-tools on their systems) I'd recommend you port your code to Python3. The 2to3 tool might be of great help here.
Kind regards, Andreas.

The following command failed:

/tmp/_MEIVWgmag/harvest --midpoint-reroot -u -q -i /project/genomicfoodbornepathogenecology/Continuum/ST58fasta/parsnp/parsnp.ggr -o /project/genomicfoodbornepathogenecology/Continuum/ST58fasta/parsnp/parsnp.ggr -n /project/genomicfoodbornepathogenecology/Continuum/ST58fasta/parsnp/parsnp.tree
Please veryify input data and restart Parsnp. If the problem persists please contact the Parsnp development team.

Messages cannot be greater than 2GB (before compression) due to the implementation of Protocol Buffers. This limit is hit with large alignments because Gingr files rely on zlib compression after serialization. There are several potential solutions:

• Reduce the serialized size by storing only variant rows rather than entire columns (also enables row-based filters)
• Write multiple protobuf messages to each Gingr file
• Use another serialization format, such as HDF5

Hi,
I am working with my 70 assembled genomes to identify core SNPs. Therefore, I am looking to use masking repeat file .bed (generated from Mummer). Please let me know how to use the file in the analysis with the following command.

parsnp -g /ref/ref_genomic -d scaffolds/*.fasta -c

Thank you!

Hi,

I have been using parSNP successfully a couple of times.
The last time I ran it on 50 K.pneumoniae genomes and it didn´t produce any error message.
But when I wanted to convert the .ggr file or .xmfa file to a mutlifasta file with harvestools it was not possible. Although I was able to open the files in ginger without problems.
The message I got is:
ERROR:LCB 634 extends beyond reference (position 47356)
When the programm tried to write the new file.
I am using the Version 1.2

Is the gingr file broken?

Thank you,
Lisa