Giter VIP home page Giter VIP logo

datamonkey-js's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

datamonkey-js's Issues

Create new queuing mechanism

Currently, queuing is done per analysis type.

We should switch to a more sophisticated type that takes into consideration:

  • IP Addresses
  • Analysis
  • Sites and Sequence number
  • Using heuristics gained from logs

[HIV TRACE] UI issues

Dear @stevenweaver,

The menu bar does not work for me (e.g. no items in the "Cluster menu") in HIV-TRACE result pages, and the template UI is several generation behind release for Secure HIV-TRACE as it looks like.

Best,
Sergei

[HIV-TRACE] runtime errors

Dear @stevenweaver,

There seem to be issues with HIV-TRACE when edge filtering or contaminant removal is turned (exceptions returned). Could you please run a few tests to confirm? If you can't, I'll show you the problematic dataset tomorrow.

We want HIV-TRACE to be running smoothly because the paper is about to come out.

Best,
Sergei

Excessive mongodb connections

datamonkey-js is not always closing mongodb connections, and is continually creating new ones. This can cause the node process to periodically restart.

Update mongo API calls to 4.x.x

234739 (node:7563) DeprecationWarning: `open()` is deprecated in mongoose >= 4.11.0, use `openUri()` instead, or set the `useMongoClient` option if using `connect()` or `createConnection()`. See http://mongoosejs.com/docs/connections.html#use-mongo-client
234740 (node:7563) DeprecationWarning: Mongoose: mpromise (mongoose's default promise library) is deprecated, plug in your own promise library instead: http://mongoosejs.com/docs/promises.html

Create job reaper

Under rare circumstances (< 1% of jobs), there will be instances where jobs will never be marked as either aborted or completed. The exact cause for this is currently not fully appreciated, but it seems to happen in concurrence with the restart of the datamonkey-js pm2 process due to an unhandled exception (e.g. failing to check if the results JSON is valid before an attempted parse with no exception handling, triggering configured system limits, etc.). We should create a periodic reaper that harvests the jobs marked as "running" for longer than a pre-determined interval and mark whether the job has been completed or didn't make it.

Allow for separate view of tree on results pages

Oftentimes, the tree will be too large to do anything meaningful within the results page itself. We should allow for the tree to be viewed as an entirely separate window. In addition, we should have more export options such as all branch attributes associated with the tree.

BUSTED does not accepted Nexus files

Both HyPhy and HyPhy vision can accept a Nexus file with partitioned data when using BUSTED (see, for instance, https://github.com/veg/hyphy-vision/blob/gh-pages/methods/busted/data/BUSTED_2partitions_testbg.json).

However, Datamonkey currently errors out when uploading a Nexus file. At a minimum, the user should be presented with the option of selecting all, internal, or leaf branches to test for selection, as would be the case running HyPhy from the command line.

Move CSV export button for HIV-trace

The CSV button is not placed in a particularly logical spot. I see two places for it

  1. The button UI bar above the network plot (easy)
  2. The fixed nav bar at the top of the page (this is where 'global' analysis exports/functions should probably go anyway).

Meme stops running in datamonkey

Hi

I´ve been trying to use MEME for the past two weeks and sometimes it runs and sometimes it doesn´t but in the log file I do not get any error or warning saying why it stops. Here is the lattest job I tried to run.
log(1).txt

Use HyPhy Vision Fonts

The fonts that were chosen by Michael Li for HyPhy Vision did not carry over to Datamonkey.

FEL analysis giving Bad Request

I was trying to do a FEL analysis with Datamonkey and even if I try with the example data set (the flu.fasta one), use defaults then select all or leaf nodes I get a "400 bad request" error (this is the link that appears http://datamonkey.org/fel/<%=%20fel._id%20%>/select-foreground, something doesn't seem kosher with that, it looks like the job ID is not getting properly interpolated?

Error: EMFILE: too many open files, uv_resident_set_memory

The following error has occurred 23 times over the past year as of the opening of this ticket :

277303 Error: EMFILE: too many open files, uv_resident_set_memory
277304     at process.memoryUsage (internal/process.js:100:5)
277305     at Object.exception.getProcessInfo (/opt/datamonkey-js/node_modules/winston/lib/winston/exception.js:33:26)
277306     at Object.exception.getAllInfo (/opt/datamonkey-js/node_modules/winston/lib/winston/exception.js:17:24)
277307     at exports.Logger.Logger._uncaughtException (/opt/datamonkey-js/node_modules/winston/lib/winston/logger.js:602:24)
277308     at emitOne (events.js:115:13)
277309     at process.emit (events.js:210:7)
277310     at process._fatalException (bootstrap_node.js:329:26)

We should investigate which file handlers are not being properly closed.

npm install breaks

@stevenweaver and @spond , nice job so far on DM-js. I'm curious to try it out, but npm install is broken on both Linux and OSX:

> [email protected] postinstall /Users/simon/Programs/datamonkey-js
> ./node_modules/.bin/lessc ./contrib/dm-bootstrap/overrides.less > ./public/assets/css/bootstrap.css

FileError: '../bootstrap/less/bootstrap.less' wasn't found in /Users/simon/Programs/datamonkey-js/contrib/dm-bootstrap/overrides.less on line 1, column 1:
1 @import "../bootstrap/less/bootstrap.less";
2 

npm ERR! weird error 1
npm ERR! not ok code 0

Admin Panel

We should create an admin panel that allows administrators to

  1. View number of jobs spawned in last day
  2. Reports cancelled and aborted jobs.
  3. Reports completed jobs.
  4. Provides links to logs of HyPhy jobs that were unable to complete. The logs should only support stack traces.
  5. Node process metrics
  6. Cancel jobs

Remove datamonkey footer image

The datamonkey image as part of the footer is a bit large on screens with smaller resolutions. We either remove it or create a proper bootstrap footer with the image, maximum height 60 px.

Consolidate results.ejs files

Results ejs files can now be a template. It should be the same for all methods besides which hyphyVision function is called.

Stats page is too slow

The stats page is currently very slow to load. If a native solution in MongoDB is not available, then we should look into caching stats in redis and serving from there.

Filter unsupported edges option

Add an option to the set-up page (selected by default) to remove unsupported edges using the triangle phylogenetic trick.

Issue with branch display with fitted tree

hello,
I have run an analysis using aBSREL that creates the desired output data, but when I hover over the branches of the fitted tree with the mouse cursor, the branch information does not appear. I have run it on different browsers but encountered the same problem. thank you for your help.

Allow users to name analyses

In order for users to remember what runs are which, we should allow them to name them.

  • On the job submission page, add a new field with a label such as "alias", "run name", etc.
  • Place the name on the page under the method title in the progress and results pages
  • Reference the name in the email as well.

[RELAX] test and reference sets reversed

With the newer version of HyPhy, it seems as though how annotated trees are parsed has changed. We should update datamonkey-js job submission to reflect his change.

### Branch sets for RELAX analysis
* Selected 43 branches as the _test_ set: `478488770, 149706207, 664699950, 664699948, Node252, Node250, Node248, 795152647, 724899480, Node259, 109068106, 635116136, 795402852, 795323436, 402864747, Node270, Node268, 795228246, 685533192, 545687090, Node275, Node273, Node267, Node265, Node263, 332224382, 297681450, 426357812, 110832709, 57114123, Node284, Node282, Node280, Node278, Node262, Node258, 296210687, 403256847, Node289, Node287, Node257, Node255, Node247`
* Selected 7 branches as the _reference_ set: `829888493, 831225769, 826351608, Node244, Node242, 640810396, 817317597`
* 353 branches are in the unclassified (nuisance) set: `12054987, 6687226, 700438741, 4027889, 17974228, Node9, 155008488, 38231558, 8671490, 183985969, 54311338, Node18, Node16, Node14, Node12, Node8, 530573866, 591386123, Node23, 884527912, 884527910, Node27, 12667704, 5565637, Node31, 327288995, 884527920, 73666470, 884527918, 884527916, 884527914, Node44, Node42, Node40, Node38, 884527908, 240254682, 602672386, 240254688, Node52, Node50, 884527922, 884527928, 884527924, 884527926, Node59, Node57, Node55, Node49, Node47, Node37, Node35, 884527906, Node34, Node30, Node26, Node22, 59895726, 59895724, Node65, 85720641, 86211179, 86211177, 612053414, Node73, Node71, Node69, 395539363, 38231560, Node77, 52857761, 52221229, 31322956, Node82, Node80, Node76, Node68, Node64, 585696530, 507555013, 507693866, 731236363, 533178379, 512989065, 289629292, Node98, Node96, Node94, Node92, 674074617, 532046007, 14192790, 625289930, Node106, 880974618, 589933402, 568939379, 568939385, 37194856, Node116, Node114, 213579634, 25742620, Node119, Node113, Node111, Node109, Node105, Node103, Node101, Node91, 852759624, 667308135, 562835163, Node125, 83630405, 532067275, Node129, 655819600, 504165095, Node132, Node128, Node124, Node122, Node90, Node88, 586461042, 634888841, Node139, 51101223, 471399281, Node142, Node138, 98956513, 617617805, Node145, Node137, 507667492, 47522990, 560959760, 560917086, 744551514, 743705125, Node159, Node157, Node155, Node153, 594657847, 593762353, 602719153, 466021507, 470597006, Node169, Node167, Node165, Node163, 1905885, 594102812, 555986909, 742236428, Node177, Node175, Node173, 556772617, 548463879, 803253841, Node183, Node181, 295007320, Node180, Node172, Node162, Node152, 591302466, 410952805, Node188, 511834482, 62275622, Node193, 671026532, 62275609, 301755248, Node198, Node196, Node192, 472356891, 73975826, Node201, Node191, Node187, Node151, 284813839, 284813829, 284813843, Node208, Node206, 641724698, 284813833, Node212, 284813841, 254838572, 558123362, 98956515, 554548891, Node222, 584053674, 284813831, 194400541, Node227, Node225, Node221, Node219, Node217, Node215, Node211, Node205, 759111738, 98956505, 586551086, 284813847, Node235, Node233, Node231, 505801345, 507931660, Node238, Node230, Node204, Node150, Node148, Node136, Node241, Node135, Node87, Node85, Node63, Node21, Node7, Node5, 698413180, 729741604, 768405188, 541992323, 529449849, Node299, Node297, Node295, Node293, 723552850, 697870545, Node304, 701343627, 700375237, 154813845, 704335932, 677991594, 727000194, 669286296, Node318, Node316, 675421290, 30408052, 543380551, 30408056, 30408054, Node329, Node327, Node325, 30408060, 734704490, 7981414, 30408058, 734704468, 734704494, 734704476, Node345, Node343, Node341, Node339, Node337, 734704466, 734704474, Node349, 734704478, 734704482, Node352, Node348, Node336, 734704488, 543297617, 115529268, 542185675, Node359, Node357, Node355, Node335, Node333, 30408062, 525028303, 30408064, Node364, Node362, Node332, Node324, 734704496, 734704508, 734704506, 734704502, 734704498, 734704500, Node375, Node373, Node371, Node369, Node367, Node323, Node321, Node315, Node313, Node311, Node309, Node307, Node303, 694850140, 697033715, 699660929, 696987556, Node384, Node382, Node380, 686608688, Node379, 45382143, 704289124, Node388, Node378, Node302, Node292, Node4, 296142098, 296142096, 296142094, Node394, Node392, 701322822, 2894432, 570517896, 296142092, Node401, Node399, Node397, Node391, Node3`

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.