veg / datamonkey-js Goto Github PK
View Code? Open in Web Editor NEWDatamonkey - Run state-of-the-art statistical models through the comfort of your browser
Home Page: http://datamonkey.org
License: Other
Datamonkey - Run state-of-the-art statistical models through the comfort of your browser
Home Page: http://datamonkey.org
License: Other
Currently, queuing is done per analysis type.
We should switch to a more sophisticated type that takes into consideration:
See attached movie. When hovering over a method menu item, the text that used to be in the tooltip is now appended permanently to the page. This could be the case for other tooltips as well.
Dear @stevenweaver,
The menu bar does not work for me (e.g. no items in the "Cluster menu") in HIV-TRACE result pages, and the template UI is several generation behind release for Secure HIV-TRACE as it looks like.
Best,
Sergei
Installation currently needs an additional npm install winston
for logging
Dear @stevenweaver,
There seem to be issues with HIV-TRACE when edge filtering or contaminant removal is turned (exceptions returned). Could you please run a few tests to confirm? If you can't, I'll show you the problematic dataset tomorrow.
We want HIV-TRACE to be running smoothly because the paper is about to come out.
Best,
Sergei
datamonkey-js is not always closing mongodb connections, and is continually creating new ones. This can cause the node process to periodically restart.
234739 (node:7563) DeprecationWarning: `open()` is deprecated in mongoose >= 4.11.0, use `openUri()` instead, or set the `useMongoClient` option if using `connect()` or `createConnection()`. See http://mongoosejs.com/docs/connections.html#use-mongo-client
234740 (node:7563) DeprecationWarning: Mongoose: mpromise (mongoose's default promise library) is deprecated, plug in your own promise library instead: http://mongoosejs.com/docs/promises.html
Some become a bit concerned when there is no new output for a while. We should provide some sort of re-assurance that the job is still running and is healthy.
Under rare circumstances (< 1% of jobs), there will be instances where jobs will never be marked as either aborted or completed. The exact cause for this is currently not fully appreciated, but it seems to happen in concurrence with the restart of the datamonkey-js pm2 process due to an unhandled exception (e.g. failing to check if the results JSON is valid before an attempted parse with no exception handling, triggering configured system limits, etc.). We should create a periodic reaper that harvests the jobs marked as "running" for longer than a pre-determined interval and mark whether the job has been completed or didn't make it.
Oftentimes, the tree will be too large to do anything meaningful within the results page itself. We should allow for the tree to be viewed as an entirely separate window. In addition, we should have more export options such as all branch attributes associated with the tree.
Both HyPhy and HyPhy vision can accept a Nexus file with partitioned data when using BUSTED (see, for instance, https://github.com/veg/hyphy-vision/blob/gh-pages/methods/busted/data/BUSTED_2partitions_testbg.json).
However, Datamonkey currently errors out when uploading a Nexus file. At a minimum, the user should be presented with the option of selecting all, internal, or leaf branches to test for selection, as would be the case running HyPhy from the command line.
The Datamonkey redesign is great. I was wondering though why you decided to get rid of the phylogenetic tree option for each position in the alignment in the output - it was a nice way to see what the ancestral state might have been...thanks and keep up the great work!!
The CSV button is not placed in a particularly logical spot. I see two places for it
Hi
I´ve been trying to use MEME for the past two weeks and sometimes it runs and sometimes it doesn´t but in the log file I do not get any error or warning saying why it stops. Here is the lattest job I tried to run.
log(1).txt
The fonts that were chosen by Michael Li for HyPhy Vision did not carry over to Datamonkey.
I was trying to do a FEL analysis with Datamonkey and even if I try with the example data set (the flu.fasta one), use defaults then select all or leaf nodes I get a "400 bad request" error (this is the link that appears http://datamonkey.org/fel/<%=%20fel._id%20%>/select-foreground, something doesn't seem kosher with that, it looks like the job ID is not getting properly interpolated?
The following error has occurred 23
times over the past year as of the opening of this ticket :
277303 Error: EMFILE: too many open files, uv_resident_set_memory
277304 at process.memoryUsage (internal/process.js:100:5)
277305 at Object.exception.getProcessInfo (/opt/datamonkey-js/node_modules/winston/lib/winston/exception.js:33:26)
277306 at Object.exception.getAllInfo (/opt/datamonkey-js/node_modules/winston/lib/winston/exception.js:17:24)
277307 at exports.Logger.Logger._uncaughtException (/opt/datamonkey-js/node_modules/winston/lib/winston/logger.js:602:24)
277308 at emitOne (events.js:115:13)
277309 at process.emit (events.js:210:7)
277310 at process._fatalException (bootstrap_node.js:329:26)
We should investigate which file handlers are not being properly closed.
@stevenweaver and @spond , nice job so far on DM-js. I'm curious to try it out, but npm install is broken on both Linux and OSX:
> [email protected] postinstall /Users/simon/Programs/datamonkey-js
> ./node_modules/.bin/lessc ./contrib/dm-bootstrap/overrides.less > ./public/assets/css/bootstrap.css
FileError: '../bootstrap/less/bootstrap.less' wasn't found in /Users/simon/Programs/datamonkey-js/contrib/dm-bootstrap/overrides.less on line 1, column 1:
1 @import "../bootstrap/less/bootstrap.less";
2
npm ERR! weird error 1
npm ERR! not ok code 0
Update stats page to display average runtimes of each model
We should create an admin panel that allows administrators to
The datamonkey image as part of the footer is a bit large on screens with smaller resolutions. We either remove it or create a proper bootstrap footer with the image, maximum height 60 px.
Create a simple API that can:
Upload
Check status
Show Result
Results ejs files can now be a template. It should be the same for all methods besides which hyphyVision function is called.
Add a Totals
tab that displays job information across all methods.
Add benchmarking statistics, such as runtime and resources used, to the results page.
The help page should also be edited and updated during transcription.
The stats page is currently very slow to load. If a native solution in MongoDB is not available, then we should look into caching stats in redis and serving from there.
Add an option to the set-up page (selected by default) to remove unsupported edges using the triangle phylogenetic trick.
The BUSTED results page currently does not correctly display the p-value for the analysis; see http://test.datamonkey.org/busted/54cfd392581018526dfe8633
Create a simple cluster of node processes for performance and reliability improvements.
test
hello,
I have run an analysis using aBSREL that creates the desired output data, but when I hover over the branches of the fitted tree with the mouse cursor, the branch information does not appear. I have run it on different browsers but encountered the same problem. thank you for your help.
In order for users to remember what runs are which, we should allow them to name them.
Create routes with example results data that can be used for local development to see how hyphy-vision results pages will look in datamonkey.
Currently, researchers can still submit jobs no matter what the job dispatcher's status is. A warning banner should be displayed if the job dispatcher is not responsive.
With the newer version of HyPhy, it seems as though how annotated trees are parsed has changed. We should update datamonkey-js job submission to reflect his change.
### Branch sets for RELAX analysis
* Selected 43 branches as the _test_ set: `478488770, 149706207, 664699950, 664699948, Node252, Node250, Node248, 795152647, 724899480, Node259, 109068106, 635116136, 795402852, 795323436, 402864747, Node270, Node268, 795228246, 685533192, 545687090, Node275, Node273, Node267, Node265, Node263, 332224382, 297681450, 426357812, 110832709, 57114123, Node284, Node282, Node280, Node278, Node262, Node258, 296210687, 403256847, Node289, Node287, Node257, Node255, Node247`
* Selected 7 branches as the _reference_ set: `829888493, 831225769, 826351608, Node244, Node242, 640810396, 817317597`
* 353 branches are in the unclassified (nuisance) set: `12054987, 6687226, 700438741, 4027889, 17974228, Node9, 155008488, 38231558, 8671490, 183985969, 54311338, Node18, Node16, Node14, Node12, Node8, 530573866, 591386123, Node23, 884527912, 884527910, Node27, 12667704, 5565637, Node31, 327288995, 884527920, 73666470, 884527918, 884527916, 884527914, Node44, Node42, Node40, Node38, 884527908, 240254682, 602672386, 240254688, Node52, Node50, 884527922, 884527928, 884527924, 884527926, Node59, Node57, Node55, Node49, Node47, Node37, Node35, 884527906, Node34, Node30, Node26, Node22, 59895726, 59895724, Node65, 85720641, 86211179, 86211177, 612053414, Node73, Node71, Node69, 395539363, 38231560, Node77, 52857761, 52221229, 31322956, Node82, Node80, Node76, Node68, Node64, 585696530, 507555013, 507693866, 731236363, 533178379, 512989065, 289629292, Node98, Node96, Node94, Node92, 674074617, 532046007, 14192790, 625289930, Node106, 880974618, 589933402, 568939379, 568939385, 37194856, Node116, Node114, 213579634, 25742620, Node119, Node113, Node111, Node109, Node105, Node103, Node101, Node91, 852759624, 667308135, 562835163, Node125, 83630405, 532067275, Node129, 655819600, 504165095, Node132, Node128, Node124, Node122, Node90, Node88, 586461042, 634888841, Node139, 51101223, 471399281, Node142, Node138, 98956513, 617617805, Node145, Node137, 507667492, 47522990, 560959760, 560917086, 744551514, 743705125, Node159, Node157, Node155, Node153, 594657847, 593762353, 602719153, 466021507, 470597006, Node169, Node167, Node165, Node163, 1905885, 594102812, 555986909, 742236428, Node177, Node175, Node173, 556772617, 548463879, 803253841, Node183, Node181, 295007320, Node180, Node172, Node162, Node152, 591302466, 410952805, Node188, 511834482, 62275622, Node193, 671026532, 62275609, 301755248, Node198, Node196, Node192, 472356891, 73975826, Node201, Node191, Node187, Node151, 284813839, 284813829, 284813843, Node208, Node206, 641724698, 284813833, Node212, 284813841, 254838572, 558123362, 98956515, 554548891, Node222, 584053674, 284813831, 194400541, Node227, Node225, Node221, Node219, Node217, Node215, Node211, Node205, 759111738, 98956505, 586551086, 284813847, Node235, Node233, Node231, 505801345, 507931660, Node238, Node230, Node204, Node150, Node148, Node136, Node241, Node135, Node87, Node85, Node63, Node21, Node7, Node5, 698413180, 729741604, 768405188, 541992323, 529449849, Node299, Node297, Node295, Node293, 723552850, 697870545, Node304, 701343627, 700375237, 154813845, 704335932, 677991594, 727000194, 669286296, Node318, Node316, 675421290, 30408052, 543380551, 30408056, 30408054, Node329, Node327, Node325, 30408060, 734704490, 7981414, 30408058, 734704468, 734704494, 734704476, Node345, Node343, Node341, Node339, Node337, 734704466, 734704474, Node349, 734704478, 734704482, Node352, Node348, Node336, 734704488, 543297617, 115529268, 542185675, Node359, Node357, Node355, Node335, Node333, 30408062, 525028303, 30408064, Node364, Node362, Node332, Node324, 734704496, 734704508, 734704506, 734704502, 734704498, 734704500, Node375, Node373, Node371, Node369, Node367, Node323, Node321, Node315, Node313, Node311, Node309, Node307, Node303, 694850140, 697033715, 699660929, 696987556, Node384, Node382, Node380, 686608688, Node379, 45382143, 704289124, Node388, Node378, Node302, Node292, Node4, 296142098, 296142096, 296142094, Node394, Node392, 701322822, 2894432, 570517896, 296142092, Node401, Node399, Node397, Node391, Node3`
FUBAR results cause datamonkey-js memory to bloat due to the JSON.parse call in fubar.getPage
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.