systemsgenetics / biodepvis Goto Github PK

Darker Orange needed for default view for edges so that the user can see single lines when viewing network.

Since we need to generalize BioDepVis to run alignments on more than two graphs, we need to make sure that the G3NA portion of the code is modular enough that it can be run several times.

Add a window legend for user.

AND : Both Terms to exists
OR: Either
NOT: Not Existing

I have updated the README instructions for rendering BioDepVis remotely using TurboVNC and VirtualGL. Currently, I can stream BioDepVis from a remote machine in the FCT lab, but it is very slow, and when I try to stream from Palmetto I get the following messages:

[VGL] WARNING: The OpenGL rendering context obtained on X display
[VGL]    :0 is indirect, which may cause performance to suffer.
[VGL]    If :0 is a local X display, then the framebuffer device
[VGL]    permissions may be set incorrectly.
[VGL] WARNING: The OpenGL rendering context obtained on X display
[VGL]    :0 is indirect, which may cause performance to suffer.
[VGL]    If :0 is a local X display, then the framebuffer device
[VGL]    permissions may be set incorrectly.
[VGL] ERROR: in readPixels--
[VGL]    398: GL_ARB_pixel_buffer_object extension not available
X Error of failed request:  GLXBadDrawable
  Major opcode of failed request:  150 (GLX)
  Minor opcode of failed request:  5 (X_GLXMakeCurrent)
  Serial number of failed request:  291
  Current serial number in output stream:  291

I'm not very familiar with OpenGL or VirtualGL so for now I leave this error log until we can debug it.

Search box appears off screen when starting G3NAV.exe if possible please have it appear in the middle of the screen. Image for reference:

Fix it for 1080p

There is a visual at http://network.genome.clemson.edu/about.php of two graphs with their conserved subgraphs. These conserved subgraphs are derived by extracting the nodes that are connected across graphs. The above visual wasn't produced by BioDepVis; however, it would probably be useful for BioDepVis to be able to visualize conserved subgraphs in the future.

Fix the bug for software crash

If we refactor BioDepVis to use the correlation matrix instead of the edge list, we can allow the user to tune the correlation threshold for edges to see how a threshold affects the graph. Note that KINC normally finds this threshold automatically, so this would mainly be for visualization rather than for finding the threshold manually. That being said, we could also compute certain properties of the resulting network when a threshold is selected (such as whether the network is scale-free) to help the user determine whether the threshold is good. This feature would require some analysis of how KINC finds the threshold automatically.

Look into Drag Functionality, Rather than an Fix Functionality

In order to be able to do "in situ" visualization (as opposed to downloading output data from remote machine), BioDepVis must be installed on the remote machine, either natively or as a container image. To do the latter we need to create a Dockerfile. The only part I'm not sure about is how to expose the VNC server through the container.

Doesn't initialize windows so that they can be moved.

Currently we load gene ontology data from a local file, go-basic-obo, which is very large to store in the repo. Eventually we should fetch the data from a remote location, or even store it in iRODS if necessary.

This API allows for data to be shared by CUDA and OpenGL, which might allow us to eliminate some memory transfers during FDL.

Large graphs with high connectivity may take a long time to visualize via FDL, so it may be useful to have a save/restore feature in which you run FDL once, save the positions to a file, and then load that file for later visualizations. This precompute step could be done without the GUI which could make it faster as well.

We are currently working to refactor BioDepVis with Qt, because Qt has a robust set of libraries to handle several of the tasks in BioDepVis. I will use this issue thread to organize our work as best I can.

Software components of BioDepVis

Here are some of the major source files:

• alignment.cpp: load alignment from file, update alignment edges
• cuda_code.cu: CUDA kernels for force-directed layout
• events.cpp: mouse / keyboard / GUI event handlers
• graph.cpp: create graph from files
• main.cpp: parse command-line arguments, create GUI
• parse.cpp: parse JSON config file
• util.cpp: global variables, a lot of the OpenGL code

Also, these source files are actually external libraries:

• Camera.cpp / Matrix.cpp / Utility.cpp / Vector.cpp
• jsoncpp.cpp
• lodepng.cpp

And some smaller source files I didn't mention:

• loader.cpp
• miscgl.cpp
• Ont.cpp
• texture.cpp

Qt Libraries to use

• QWidget: single-window GUI, keyboard legend, gene ontology searching
• QOpenGLWidget: 3D rendering of the graphs (part of main window)
• QJsonDocument: parsing JSON configuration file

I have code for the force-directed layout in both 2 and 3 dimensions, and the 3D version is not significantly slower. Depending on what the end users want, we should either scrap one of the versions, or provide a command-line or config option to use one or the other.

The code that selects a node on with a double-click is slightly off depending on the location of the click in the GL viewport. As far as I can tell the code is identical to the original code, so I can't figure out why it's off.

On attempt to compile, I ran into math include errors, all errors where removed when I locally added #include <cmath> into the file: graph.h

anaconda/4.0.0 and python/2.7.6 are no longer available as modules on Palmetto. I tried with anaconda/4.2.0 and python/2.7.6 instead as follows:

module load python/2.7.6
module load anaconda/4.2.0
conda create --name biodepviz pyqt

y

which results in:

IOError: [Errno 13] Permission denied: u'/software/anaconda/4.2.0/pkgs/dbus-1.10.10-0/libexec/dbus-daemon-launch-helper'

Is root access needed to install these packages without building my own conda environment?

I also am having trouble with the qsub command listed in the ConfigGeneratorReadme:

[wpoehlm@user001 ~]$ qsub -X -I -l select=1:ncpus=8:mem=16gb:ngpus=1:gpu_model=k40,walltime=2:30:00
qsub: DISPLAY not set

Currently a graph is represented as three files:

1. edge list
2. node/module list
3. ontology info

We should condense these files into two files:

1. node list (id, module id, ontology terms)
2. edge list

This representation would significantly decrease load time because the ontology info file has a lot of information that is not used by the graph.

I am able to get G3NAV to start running, but it hangs and runs for over an hour without finishing.

Environment:

qsub -I -l select=1:ngpus=1:ncpus=16:mem=32gb,walltime=02:00:00
module load gcc/4.8.1 cuda-toolkit/7.5.18
export DISPLAY=:0

command:

./G3NAV.exe input.json

Output:

1
File Opened successfully!!!. Reading data from file into array
Parsed Successful {type=[7], size=2}string(graph1)
<------------>ID : 1
Name : Maize
Loc: ./data/M.tab
Cluster : ./data/M.tab.cluster
Ont: ./data/Maize_info.txt
X: -100
Y: 0
Z: 0
W: 200
H: 200
Reading Maize Filename : ./data/M.tab
--Total Nodes : 2073
--Total Edges : 33398
Count = 33398 vs 33398
Graph Size :2073 for GPU
Malloc Edge Matrix
Malloc Node Position
Malloc Node Prop
Mem Cpy Node Position
Mem Cpy Node Prop
Mem Cpy Edge
Setup Complete
Load Complete
string(graph2)
<------------>ID : 2
Name : Rice
Loc: ./data/R.tab
Cluster : ./data/R.tab.cluster
Ont: ./data/Rice_info.txt
X: 100
Y: 0
Z: 0
W: 200
H: 200
Reading Rice Filename : ./data/R.tab
--Total Nodes : 2259
--Total Edges : 32821
Count = 32821 vs 32821
Graph Size :2259 for GPU
Malloc Edge Matrix
Malloc Node Position
Malloc Node Prop
Mem Cpy Node Position
Mem Cpy Node Prop
Mem Cpy Edge
Setup Complete
Load Complete
string(alignment1)
Perform Alignment on 1 2 using ./data/magma0.5.output
--Total Alignment Edges : 20732073,2073
Matrix Alloc Complete
---><--Edge Count Verified : 2073 vs 2073
Malloc Align Edge Matrix
Mem Cpy Edge Align
Setup Complete
Edges Moved: 2073

It seems to hang after the last "Edges Moved" line, without finishing or producing an error message

Compiled program on cluster and connected successfully to graphics node using turboVNC. When I run the command G3NAV.exe, I get the following:

Xlib: extension "GLX" missing on display ":1".
freeglut (./G3NAV.exe): OpenGL GLX extension not supported by display ':1'

Now that BioDepVis has been refactored, we can begin to think about additional features. This depends mainly on what the end users want, and so we may need to give them time to play with the visualization and see what they think of. Some ideas that have been discussed so far:

• Karan's "ROI mode", which allows you to select two clusters and just display them and their corresponding alignments
• Some way to "filter" the graphs so that only selected nodes (which could be based on a search query or a selected GO term, etc.) and their alignment edges are shown
• Search term operators such as AND, OR, and NOT

I am trying to run G3NAV without any visualization. After compiling, I launched an interactive job and tried to execute G3NA with the test json file as follows:

qsub -I -l select=1:ngpus=1:ncpus=16:mem=32gb,walltime=02:00:00
cd /zfs/feltus/wpoehlm/BioDep-Vis/
module load gcc/4.8.1 cuda-toolkit/7.5.18

./G3NAV.exe input.json

I receive the following error:

1
freeglut (./G3NAV.exe): failed to open display ''

AND : Both Terms to exists
OR: Either
NOT: Not Existing