Comments (8)
Forgot to mention the version of miniasm is 0.2-r128
from miniasm.
Update:
I used the new version 0.3_r179. It seem like when data use small memory, such as 80GB, it works fine. However, when data needs big memory, it still fail with segmentation fault. And I was able to repeat the error and noticed that both time the error happened when memory usage reach to about 120GB, yet I have total 240GB memory.
......
[M::main] CMD: minimap2 -x ava-pb -t 28 /efs/fasta/merged_BAC_PACBIO.filtered.fasta /efs/fasta/merged_BAC_PACBIO.filtered.fasta
[M::main] Real time: 30832.901 sec; CPU: 674449.293 sec
31699 Segmentation fault (core dumped) /home/biodocker/miniasm//miniasm -f /efs/fasta/merged_BAC_PACBIO.filtered.fasta /efs/fasta//miniasm/BAC_corn.paf.gz -Rc2 > /efs/fasta/miniasm/BAC_corn.gfa 2> /efs/fasta//miniasm/step2.err
Is there any other way to decrease the memory useage?
Best,
Xin
from miniasm.
I run into same error when using miniasm with large datasets. In case, it consistently fails around 256GB memory beyond a certain file size.
from miniasm.
same here, I get the Segmentation fault error when the process uses about half of the total memory (in my case ~65 GB out of 126). My version is 0.3-r179 as well.
from miniasm.
@lh3 does miniasm have a upper limitation of what it can actually read? I am trying to assemble a gzipped 4Tb PAF file and it looks there is a hard cap at around 2Tb memory usage.
from miniasm.
We are experiencing this as well. we have 1TB of RAM and the segfault consistently occurs around the halfway point like others here; which is around of 550GB used for us. We also use a gzipped paf file. We were using 0.2-r168-dirty and updated to the current release and the issue still occurs.
from miniasm.
@mr--white set -m
to something between 1000-2000 during your minimap2 overlap. Resulting PAF file should much smaller and miniasm run through.
from miniasm.
am having the same problem anyone tried out what at @mr--white suggested to see if it worked?
am working with a 734 GB .paf.gz file, wouldnt want to repeat this step if it doesent work.
from miniasm.
Related Issues (20)
- using a fragmented genome as an input
- how to calculated identity? HOT 1
- No output in GFA file after miniasm run. HOT 4
- Explanation of GFA file format HOT 1
- PAF end coordinates HOT 2
- Segmentation fault (core dumped) while running miniasm HOT 1
- Can miniasm be used for contigs HOT 1
- Hello there, HOT 3
- Number of rounds for short overlap removal
- miniasm prints no gfa if PAF file contains self-mapped read HOT 3
- Conceptual Question About Miniasm
- Empty gfa file after miniasm run HOT 4
- Decreasing minimum required read coverage for assembly
- Miniasm killed process
- PAF Format detail HOT 1
- Not results generating? HOT 5
- Empty paf file. No error message.
- The main disadvantages of Miniasm compared to more modern assemblers
- combine replicates
- Assemble of DNA and RNA amplicons from PacBio run HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from miniasm.