compevol / beast2 Goto Github PK
View Code? Open in Web Editor NEWBayesian Evolutionary Analysis by Sampling Trees
Home Page: www.beast2.org
License: GNU Lesser General Public License v2.1
Bayesian Evolutionary Analysis by Sampling Trees
Home Page: www.beast2.org
License: GNU Lesser General Public License v2.1
Hi everyone
Is there a possibility to perform tip date sampling with individual priors in the Beast2 package? I know this option from Beauti 1.7.5. In Beauti2 however, it seems to me that I can only give the same prior distribution to all samples within a taxon set.
Thanks for your help
Amelie
Some plugins (RandomTree, CalibratedYuleModel, maybe others) pluck inputs from the prior (with what looks to me like risky code) or from their outputs.
It is not clear to me how this information is supposed to reach users who wish to modify the XML or even developers which may miss the code.
Should we really design plugins this way? I thought the whole idea in having "inspectable" inputs was to eliminate magic.
(I scratched my head in disbelief several times when I removed the inputs to RandomTree from the XML, yet they appeared in the debugger)
Print out the full name of the plugin in an error message, since two plugins with the same name may exist (like Uniform and Uniform, distribution and operator)
I tried running the latest version of Beauti with an old version of the RBS addon.
This resulted in an exception, and no user level error message.
Would be nice to have some versioning information which enable Beauti to detect this on startup and issue a sane, user-level warning or failure.
Port Sibon Li's model averaging of relaxed clock models from beast1 to beast2
Hi Walter
I have run into another problem while using Beauti2.
When I try to import the attached nexus file, I get an error saying # taxa is not equal to dimension.
However, the same file works perfectly well in Beauti 1.75.
Any help would be appreciated.
Regards,
Alex
Sep 29, 2013
Project Member #1 higgs.ml
This happens when the number of sequences found in the nexus file does not equal the number specified in the NEXUS dimensions block, e.g.
DIMENSIONS NTAX =26 NCHAR=614;
This should at least result in a warning, since it may be the alignment is malformed.
Do we have an example nexus file to reproduce this?
Remco
The Nexus parser (beast.util.NexusParser) only handles a minimum subset of the Nexus format. I suggest we switch to the JEBL nexus parser which is much more complete (handles Taxa and Characters blocks etc.). The wrapper class beast.app.beauti2.util.BeautiParser shows how to use this to load multiple partitions from a nexus file.
Dear Walter,
Can you add this as an issue on BEAST2 google code site? The fix should be to automatically create the directory if it doesn't exist (and report doing so on the terminal).
Cheers
Alexei
Begin forwarded message:
Hi Yoshi,
The path sampler creates subdirectories, one for each step and the
rootdir='/tmp/step' in the example file specifies the locations of these
files. The error message you see indicates that the directory /tmp/step
does not exist on your computer.
You can either change the file and point rootdir to another location
that does exist on your computer -- and where you want the files to go.
Alternatively, you can create a /tmp/step directory on your computer.
Hope this helps,
Remco
I start with the largest possible -Xmx my 32bit cpu/java allows on my laptop. The trees file "only" 340M, and Remco says he successfully resumed larger files.
The file can be downloaded from
https://dl.dropboxusercontent.com/u/5675908/conus-al.tar.bz2
java -ea -Xmx2500m -Xss2024k -jar ... -seed 666 -resume conus-al.xml
...
Start likelihood: -30613.211722592572
Appending file conus-al.log
Sample posterior ESS(posterior) likelihood prior
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuilder.append(StringBuilder.java:119)
at beast.core.Logger.openLogFile(Unknown Source)
at beast.core.Logger.init(Unknown Source)
at beast.core.MCMC.run(Unknown Source)
at beast.app.BeastMCMC.run(Unknown Source)
at beast.app.beastapp.BeastMain.(Unknown Source)
at beast.app.beastapp.BeastMain.main(Unknown Source)
User report "Is there any reason that the “drag-and-drop” method of importing alignments into BEAUti has gone away? I find this really handy as I’m selecting from a giant folder with hundreds of NEX files. Perhaps this feature could be returned in a future bug fix."
So Remco can divulge himself of the duty.
Resumption of MCMC expected, instead failure to resume can sometimes occur:
Error message from Thomas Vignaud:
Start likelihood: -7892.4621158294585
java.lang.Exception: Error 400: Cannot resume: log files do not end in same sample number
at beast.core.Logger.openLogFile(Unknown Source)
at beast.core.Logger.init(Unknown Source)
at beast.core.MCMC.run(Unknown Source)
at beast.app.BeastMCMC.run(Unknown Source)
at beast.app.beastapp.BeastMain.(Unknown Source)
at beast.app.beastapp.BeastMain.main(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at apple.launcher.LaunchRunner.run(LaunchRunner.java:116)
at apple.launcher.LaunchRunner.callMain(LaunchRunner.java:51)
at apple.launcher.JavaApplicationLauncher.launch(JavaApplicationLauncher.java:52)
A minimum amount of parsing ability is required to extract taxa, sequences, dates and attributes from the XML files of BEAST 1.x. An importer for these is already available in dr. app.beauti.utils.
This would be a great benefit for the transition between versions.
load nex (https://gist.github.com/tgvaughan/657cbfc3761e3fb7a009) and use *BEAST template, and guess trait. The XML in species part has wrong mapping (s0_tip11, s0_tip01) that are not existing in nex and BEASUti GUI.
<taxonset id="taxonsuperset" spec="TaxonSet">
<taxon id="s0" spec="TaxonSet">
<taxon id="s0_tip1" spec="Taxon"/>
<taxon id="s0_tip11" spec="Taxon"/>
<taxon id="s0_tip0" spec="Taxon"/>
<taxon id="s0_tip01" spec="Taxon"/>
</taxon>
<taxon id="s2" spec="TaxonSet">
<taxon id="s2_tip0" spec="Taxon"/>
<taxon id="s2_tip01" spec="Taxon"/>
<taxon id="s2_tip1" spec="Taxon"/>
<taxon id="s2_tip11" spec="Taxon"/>
</taxon>
<taxon id="s1" spec="TaxonSet">
<taxon id="s1_tip1" spec="Taxon"/>
<taxon id="s1_tip11" spec="Taxon"/>
<taxon id="s1_tip0" spec="Taxon"/>
<taxon id="s1_tip01" spec="Taxon"/>
</taxon>
<taxon id="s3" spec="TaxonSet">
<taxon id="s3_tip1" spec="Taxon"/>
<taxon id="s3_tip11" spec="Taxon"/>
<taxon id="s3_tip0" spec="Taxon"/>
<taxon id="s3_tip01" spec="Taxon"/>
</taxon>
<taxon id="s4" spec="TaxonSet">
<taxon id="s4_tip1" spec="Taxon"/>
<taxon id="s4_tip11" spec="Taxon"/>
<taxon id="s4_tip0" spec="Taxon"/>
<taxon id="s4_tip01" spec="Taxon"/>
</taxon>
</taxonset>
Idea for an enhancement that would allow substbma and similar sitemodels to be just sitemodels, rather than also implementing their own TreeLikelihoods.
Reported by shvetanks, Jun 12, 2013
File should be listed in Partitions with details of sequences. Instead the following error shows- "Something went wrong importing the alignment:
Around line 2549
Unknown code found in sequence: X"
Error in Beauti 2.0.2 on Mac OSX 10.8.3
This is an amino acid multiple alignment file.
Changing the extension to .nex or .nexuss after conversion does not help
Sep 11, 2013
Project Member #1 dong.w.xie
Could you please attach the example of nex file? I cannot replicate using our example.
Thanks,
Walter
Add comments to code that is not obvious, rename variables to conform to Java naming standard, and generally clean up code
Relative operator weights would make life easier for *BEAST implementers.
My proposal is straightforward.
Note: not valid beast2 XML, but you get the idea
<operator weight="10">
<operator name="op1" weight="2"/>
<operator name="op2" weight="1"/>
</operator>
<operator name="op3" weight="5">
This is equivalent to the weights (10/15 * 2/3),(10/15 * 1/3) for op1/op2 and 5/15 for op3.
Basically, the total sum of all operators grouped together is equal to the weight of the group. It would be nice to have a democratic default:
<operator weight="10">
<operator name="op1"/>
<operator name="op2"/>
</operator>
is equivalent to a weight of 5 for both.
Project Member #1 higgs.ml
Note, these meta-operators need care to store/restore from file (when resuming).
Sep 30, 2013
Project Member #4 higgs.ml
Issue 20 has been merged into this issue.
Cc: jheled
Oct 8, 2013
Project Member #5 higgs.ml
support for nested operator info in state file is now available
The initAndValidate() for the CalibratedYuleModel class contains this code:
if (nCals == 0) {
// assume we are in beauti, back off for now
return;
}
Is this really the best BEAST2 can do for developers. How can a developer know if this magic is needed, and where? if there is no alternative, this must be very prominently and thoroughly documented.
I see this code in ClusterTree.java
public void initAndValidate() throws Exception {
if (Boolean.valueOf(System.getProperty("beast.resume")) &&
(isEstimatedInput.get() || (m_initial.get() != null && m_initial.get().isEstimatedInput.get()))) {
...
}
Is that the best we can do for beast2 implementers?
If so, it should be prominently documented (I was not aware of this issue until now).
It would be nice if the code knew not to call the initAndValidate on resume, or maybe just the parts that matter? Perhaps we can separate the init into two parts (first-time and resume)? or at least pass a flag in?
This is related to issue 32 .
I have replaced most of "Add-on(s)" into "Plugin(s)" in wiki, except of 2 wiki pages that contain GUI screenshots and user guide:
BEAUti 2 needs to be updated first, before these 2 pages can be re-written.
Walter
Sep 13, 2013
Project Member #1 alexei.drummond
I prefer "Package" to "Plugin". Package sounds like something you could write a paper about and be proud of (as with an R package).
Once we decide on the right name, it should be applied everywhere, not piecemeal.
Sep 22, 2013
Project Member #2 alexei.drummond
(No comment was entered for this change.)
Expected to be able to specify location for taxa.
There is only a single dummy taxon.
Needs to make all taxa in species tree available.
add a unit test that tests that the ScaleOperator correctly scales a tree
Refactoring to allow flexibility in data representation.
Hi Walter
I was following the Phylogeography example that is on the BEAST2 page. I installed the add-ons BEAST_CLASSIC and BEASTii. I then click the + at the bottom left of the partition tab and choose to add continuous geography. I use this to enter the latitude and longitude for my samples.
When I followed this for the phylogeography example (raccoon rabies virus I believe) it worked fin and I could save the XML file.
Regards,
Alex
Project Member #1 dong.w.xie
Hi Walter
I fixed the problem, it seemed to be occurring because I was using & to seperate groups.
When I changed all instances of & to _ it saved without any errors.
Regards,
Alex Stuckey
input original tree below into newick tree TreeParser, and then BEAST 2 output a different tree as inputted.
BEAST 2 output:
0.036831999999999976,1217:(615:(613:(612:(611:(2:0.0,3:0.006647999999999987):0.09753100000000003
input:
0.036831999999999976,(((('NZAC03011539|Plot_4|Subplot_P|LBIinv0013|G6|MT-CO1':0.0,'NZAC03013498|Plot_5|Subplot_A|LBIinv0012|B2|MT-CO1':0.006647999999999987):0.09753100000000003,
the number of brackets increased from 4 to 5.
original tree is attached.
Project Member #1 alexei.drummond
Not sure this is a bug with TreeParser class. We should focus on issues with BEAST2, rather than code that uses BEAST2 classes.
Status: New
Labels: -Priority-High Priority-Low
CalibratedYuleModel.initAndValidate() contains this code
if (!_MRCAPrior.isMonophyleticInput.get()) {
throw new Exception("MRCAPriors must be monophyletic for Calibrated Yule prior");
}
and
if (_MRCAPrior.isMonophyleticInput.get()) {
System.err.println("WARNING: MRCAPriors must have a distribution when monophyletic for Calibrated Yule prior");
}
I hope the exception is caught by BEAST2 and passed in a nicer way to the user (but not sure).
But the System.err.println looks bad to me. How does a plugin issues a warning? I think We need a system wide facility.
Project Member #2 higgs.ml
Is this an error condition or should this be treated as warning?
Sep 29, 2013
Project Member #3 jheled
Since I can't remember adding this part of the code, I am assuming it is one of your additions. I would rather not have any of this "magic" and always require the calibration points to be specified explicitly.
My report was more on the topic of error reporting by a plugin.
A user may want to add other monophyly constraints. This breaks the prior guarantee, but the code can still run, so my guess it is just a warning.
RandomTree.initAndValidate() calls its own initStateNodes()
Seems like a wasted action, since initStateNodes() is called from the main initialization code.
I tried a plain 'ant' and got into trouble, which might have been avoided if the default was 'addon'
(but maybe this is not possible??)
Project Member #1 higgs.ml
This is more an issue for addons than for the core?
Not sure what can be done in the core to fix this.
Sep 29, 2013
Project Member #2 jheled
Perhaps we need to separate core issues and non-core issues?
As far as I understand, this issues list is for the whole of beast2 and not just the core.
I am assuming we will have a detailed example and/or documentation on how to build an addon (AKA the new plugin), and I was hoping the default there would be 'ant addon'
An init node in the state without an ID seems to run fine, but cause a mysterious "Null pointer exception" on resume.
If this is really serious, the program should be flagged as bad before running?
Project Member #1 higgs.ml
Init nodes should not pass on their ID to the node they initialise.
Status: Accepted
Labels: -Priority-Medium Priority-High
Oct 8, 2013
Project Member #2 higgs.ml
This might be fixed with the new implementation of storeToFile using JSON.
Choose whether to add new methods to Tree class or add subclass SampledAncestorTree.
This is a crucial development to allow SubstBMA plugin (published in 2012) to be used/cited in BEAST2.
Project Member #3 higgs.ml
subst-bma plugin needs to be compilable
Oct 2, 2013
Project Member #4 alexei.drummond
Subst-bma compiles against BEAST2.0.2 -- if BEAST2.1 has changed the API such that subst-bma doesn't compile then I guess this issue depends on first changing it so it compiles again?
Oct 2, 2013
Project Member #5 higgs.ml
That is what I would expect.
If you foresee any development for v2.0.2, just branch off in SVN for that version only.
When editing a value of a parameter -- which is a list of primitives -- in the init panel, nothing suitable is found. A table editor matching dimensions would be nice.
This development involves building a Swing editor for Matrix parameters to allow easy input of initial values in BEAUti 2. Especially useful for SubstBMA plugin, but should be of general value.
TreeParser is rather too complex and has a very large number of interacting Inputs. It should be rationalised, perhaps by introducing a subclass. One key distinction is whether the labels in the Newick tree are considered to be node numbers of labels. For example, this could be separated into a superclass that only handles numbers and a subclass that handles labels.
TreeParser should also have an option for single-child nodes of either reading them in as single-child nodes (i.e. for purposes of beast-graphics or TreeStat2) or as two-child nodes with a zero branch to labelled child (i.e. for BEAST MCMC of sampled ancestor trees).
Joseph has devised a new heuristic for creating a starting state for a *BEAST analysis when there are a large numbers of loci. We should incorporate this into the *BEAST state initialisation in BEAST2 to avoid very long burn-in times with large analyses.
Project Member #1 jheled
Code submitted, tested lightly. Needs more testing, and integration with templates and BEAUTI.
detail at email "Bundling Java apps for 10.8+"
Hey Carlos,
Yes... that is a bug in Beauty I think... I got the same issue... It creates new elements using "id" (and adds 1 to pre-existing taxa names) instead of referring directly to those using "idref"
In my case I had to replace:
<taxonset id="Ingroup" spec="TaxonSet">
<taxon id="aiensis1" spec="Taxon"/>
<taxon id="cynocephalus1" spec="Taxon"/>
<taxon id="eudentatus1" spec="Taxon"/>
<taxon id="franouxi1" spec="Taxon"/>
<taxon id="hageni1" spec="Taxon"/>
<taxon id="japonicus1" spec="Taxon"/>
<taxon id="lagocephalus1" spec="Taxon"/>
<taxon id="lenguru1" spec="Taxon"/>
<taxon id="lividus1" spec="Taxon"/>
<taxon id="longifilis1" spec="Taxon"/>
<taxon id="marquesensis1" spec="Taxon"/>
<taxon id="ouwensi1" spec="Taxon"/>
<taxon id="pugnans1" spec="Taxon"/>
<taxon id="punctissimus1" spec="Taxon"/>
<taxon id="rapa1" spec="Taxon"/>
<taxon id="sarasini1" spec="Taxon"/>
<taxon id="spBIF1" spec="Taxon"/>
<taxon id="stimpsoni1" spec="Taxon"/>
<taxon id="stiphodonoides1" spec="Taxon"/>
</taxonset>
by this
<taxonset id="Ingroup" spec="TaxonSet">
<taxon idref="aiensis"/>
<taxon idref="cynocephalus"/>
<taxon idref="eudentatus"/>
<taxon idref="franouxi"/>
<taxon idref="hageni"/>
<taxon idref="japonicus"/>
<taxon idref="lagocephalus"/>
<taxon idref="lenguru"/>
<taxon idref="lividus"/>
<taxon idref="longifilis"/>
<taxon idref="marquesensis"/>
<taxon idref="ouwensi"/>
<taxon idref="pugnans"/>
<taxon idref="punctissimus"/>
<taxon idref="rapa"/>
<taxon idref="sarasini"/>
<taxon idref="spBIF"/>
<taxon idref="stimpsoni"/>
<taxon idref="stiphodonoides"/>
</taxonset>
Project Member #1 higgs.ml
What should be done to reproduce this one?
Remco
Perhaps there is a way to make addition of clades for calibration more obvious?
I missed the small '+' button in the Priors panel.
I tested that BEAUti has problem to run when the number of loci > 128 or 256.
Some users also report this, especially for *BEAST: https://groups.google.com/forum/#!topic/beast-users/HsifLjGyXfg
We discussed this in the meeting, but it seems not logged.
So that partitions can be merged as well as split. For example the examples/primates-mtDNA.nex has 5 partitions specified, but in some instances you may want to just combine the partitions into a single one if you are using a single model, and linking them would not compress the patterns fully.
The rates should be changed so as to keep the involved branch lengths in substitutions constant or near-constant after operation. Good luck with Hastings ratio.
Swing based GUI development is specialised work and tedious.
E.g. try to implement the equivalent of this in Swing:
<button value='Click me' onclick='alert("Hello world")'>
This is a hinderance for BEAUti template developemnt, which rarises the question: is there a better way to develop/extend BEAUti. One alternative is JavaFX, another HTML/Javascript/CSS.
This design change will allow for future developments including:
(1) Wrappers of Tree for use in tree prior calculations (i.e. NormalizedTree for Jessie Wu's substitution model averaging plugin)
(2) CompoundParameters that are made up of multiple parameters (including random numbers of parameters).
Project Member #4 alexei.drummond
I have completed the Parameter part of this issue. Actually the two parts are quite separate, so probably should have been two issues.
Status: Started
Sep 30, 2013
Project Member #5 higgs.ml
Dilemma: CompoundParameters either have a type, since Parameters are identified by their type, and each type can have its own interface --
RealParamterInterface, IntegerParameterInterface, etc. -- or we do away with all typed parameters. Is there another way?
Currently, very few Parameter-Inputs are available, most of them are typed parameter inputs, so a generic CompoundParameter is not very useful yet.
Oct 2, 2013
Project Member #6 alexei.drummond
Not sure what comment #5 is about? Parameter has already been changed to an interface... are more interfaces needed?
Oct 2, 2013
Project Member #7 higgs.ml
Imagine a CompoundParameter and an Operator that works on that CompoundParameter, say a ScaleOperator. Scaling only works with RealParameters, so it has a Input as an Input. To change this to CompoundParameter, either CompoundParameter has to
Any other solutions?
Oct 2, 2013
Project Member #8 alexei.drummond via email
Option 3 the nicest, no?
Oct 2, 2013
Project Member #9 higgs.ml
If we go for Option 3, we miss out on type safety, which means we could amalgamate RealParameter, IntegerParameter and BooleanParameter into Parameter.
The user should be able to link parameters in the prior panel like rateAC and rateAT parameters.
Error: cannot use beast 2.0.2 after installed 2.1.0 packages.
All versions are using the same library path.
We should create one sub-folder named by version number for each version, such as /Library/Application Support/BEAST/2.1.0/.
Hi guys,
First of all, congratulations to the fabulous work done with the new version of Beast... Every time I come back to analyses after a few months of field or lab work, I am amazed by new developments...
Just a comment, I tried to link a morphological character to a species tree (in a *Beast framework)... and it seems that Beauty always creates a new state for missing data (I code them using question marks)... I indeed get this:
<userDataType codeMap="?=0,no=1,yes=2,? = 0 1 2 " codelength="-1" id="traitDataType.crenulated" spec="beast.evolution.datatype.UserDataType" states="3"/>
I tried to edit the xml by replacing the above code by:
<userDataType codeMap="no=0, yes=1, ? = 0 1" codelength="-1" id="traitDataType.crenulated" spec="beast.evolution.datatype.UserDataType" states="2"/>
And I also updated dimensions in the substitution model...
But it generates an error from the Beast:
"Error 110 parsing the xml input file
validate and intialize error: Could not find code ? in codemap"
Am I missing something?
Proper handling of missing data is important to me, because I often (need) exclude the outgroup state from the analysis...
Thanks in advance!
Sep 1, 2013
Project Member #1 dong.w.xie
it needs further investigation to see if this is duplicated to issue #37
There are a number of MCMC algorithms (eg BDSIR, PMMH) which employ stochastic integration to estimate a component of the target distribution. While it is possible to implement these schemes within the current BEAST framework, it isn't particularly easy.
For instance, in BDSIR the stochastic component is the tree likelihood, which requires simulation of a new epidemic trajectory whenever new epidemiological parameters are proposed. Denise has had to implement this by making the trajectory an explicit component of the state and creating a special wrapper operator around each of the parameter operators which updates this state whenever the parameters are modified.
A far easier approach would be to have the likelihood perform the trajectory simulation itself and keep the trajectory out of the state entirely. However, doing this currently results in BEAST errors due to the fact that the likelihood consistency checks fail.
It would therefore be handy to be able to switch this check off for particular calculation nodes marked as "stochastic". This isn't completely trivial, as the existing consistency check involves marking all state nodes as dirty and propagating this dirtiness through the calculation node dependency graph. It's therefore the dirtiness propagation that needs modifying in the particular instance that the dirtiness is due only to the consistency check.
Port Jessie Wu's microsatellite models in BEAST1 to BEAST2
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.