options to

• set the initial sorting criteria
• set the initial grouping criteria
• aggregate all groups by default
• initially collapse the side panel
• disable adding new columns
• hide the side panel at a whole
• easier way to manipulate the initial columns (renderer type, width, ...)
• taggle overview by default config option
• simple option to trigger a specific filter dialog

• filter item with single item -> dedicated filter
  ->
  • taggle overview by default config option
  • flackern (up / down) in Firefox in OverviewRankingView (Firefox version: 52.5.0 (32-Bit))
  • very slow changing size in Firefox in OverviewRankingView

• variable Höhe von LineUp dynamisch anpassen
  -> für jede Zeile bestimmen, welche Höhe es haben
• extra overlay or fat tooltip for overlay data
• wenn views links auf anderen Gene haben, wie damit ungegeben werden sollen
  -> open in _blank oder intern oder overlay

• Release number or git hash: 0cb8aef
• Web browser version and OS: Chrome Version 62.0.3202.75 (Official Build) (64-bit) and Ubuntu 17.10
• Environment (local or deployed): local

Steps to reproduce

1. Setup any workspace, e.g. a simple tutorial workspace
2. Add caleydo/tdp_dummy to the workspace
3. Test API subpath count: http://localhost:8080/api/tdp/db/dummy/a/count

Observed behavior

• Any unexpected output or action (or lack of expected output or action)
• Web browser console errors (including tracebacks)
• Server errors (relevant messages and tracebacks)

api_1          | 14:50:57 INFO tdp_core.db: SELECT count(d.*) as count FROM b d  
api_1          | 14:50:57 INFO tdp_core.db: removing session again
api_1          | Traceback (most recent call last):
api_1          |   File "/usr/local/lib/python2.7/site-packages/gevent/pywsgi.py", line 884, in handle_one_response
api_1          |     self.run_application()
api_1          |   File "/usr/local/lib/python2.7/site-packages/geventwebsocket/handler.py", line 88, in run_application
api_1          |     return super(WebSocketHandler, self).run_application()
api_1          |   File "/usr/local/lib/python2.7/site-packages/gevent/pywsgi.py", line 870, in run_application
api_1          |     self.result = self.application(self.environ, self.start_response)
api_1          |   File "/usr/local/lib/python2.7/site-packages/werkzeug/contrib/fixers.py", line 152, in __call__
api_1          |     return self.app(environ, start_response)
api_1          |   File "phovea_server/phovea_server/dispatcher.py", line 62, in __call__
api_1          |     return app(environ, start_response)
api_1          |   File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 2000, in __call__
api_1          |     return self.wsgi_app(environ, start_response)
api_1          |   File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1991, in wsgi_app
api_1          |     response = self.make_response(self.handle_exception(e))
api_1          |   File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1567, in handle_exception
api_1          |     reraise(exc_type, exc_value, tb)
api_1          |   File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app
api_1          |     response = self.full_dispatch_request()
api_1          |   File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request
api_1          |     rv = self.handle_user_exception(e)
api_1          |   File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception
api_1          |     reraise(exc_type, exc_value, tb)
api_1          |   File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request
api_1          |     rv = self.dispatch_request()
api_1          |   File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request
api_1          |     return self.view_functions[rule.endpoint](**req.view_args)
api_1          |   File "/usr/local/lib/python2.7/site-packages/flask_login/utils.py", line 228, in decorated_view
api_1          |     return func(*args, **kwargs)
api_1          |   File "/phovea/tdp_core/tdp_core/sql.py", line 114, in get_count_data
api_1          |     r = db.get_count(database, view_name, request.args)
api_1          |   File "/phovea/tdp_core/tdp_core/db.py", line 362, in get_count
api_1          |     r = sess.run(count_query.format(**replace), **kwargs)
api_1          |   File "/phovea/tdp_core/tdp_core/db.py", line 133, in run
api_1          |     result = self._session.execute(parsed, kwargs)
api_1          |   File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 1034, in execute
api_1          |     bind, close_with_result=True).execute(clause, params or {})
api_1          |   File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
api_1          |     return meth(self, multiparams, params)
api_1          |   File "/usr/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
api_1          |     return connection._execute_clauseelement(self, multiparams, params)
api_1          |   File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
api_1          |     compiled_sql, distilled_params
api_1          |   File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
api_1          |     context)
api_1          |   File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
api_1          |     exc_info
api_1          |   File "/usr/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 200, in raise_from_cause
api_1          |     reraise(type(exception), exception, tb=exc_tb, cause=cause)
api_1          |   File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
api_1          |     context)
api_1          |   File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
api_1          |     cursor.execute(statement, parameters)
api_1          | OperationalError: (sqlite3.OperationalError) near "*": syntax error [SQL: u'SELECT count(d.*) as count FROM b d  ']
api_1          | {'GATEWAY_INTERFACE': 'CGI/1.1',
api_1          |  'HTTP_ACCEPT': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8',
api_1          |  'HTTP_ACCEPT_ENCODING': 'gzip, deflate, br',
api_1          |  'HTTP_ACCEPT_LANGUAGE': 'en-US,en;q=0.9',
api_1          |  'HTTP_CACHE_CONTROL': 'no-cache',
api_1          |  'HTTP_CONNECTION': 'close',
api_1          |  'HTTP_COOKIE': 'randomCredentials=cranky_golick@b37hyd; session=.eJwdzr1qwzAUQOFXKXfOEP8thgwGFWPDvUZBsrhaQus4jaWog5PQSiHv3tDpfON5wOG0ztcz1Lf1Pm_gsByhfsDbJ9TArnfY6mgD_6BoIpu9I-cLVl85mfGC5j2iGR2GfmE3ZeyawoouQ9N7fJmS3g6iq0hgiUFGq7jgsPfWyF9SumCjIyWO5OR2UBdvXVOhmEp054WMrijJSGrKqeWEqctR4asyQzX6QchyaGXkJEsbcAfPDdyv8_r_Dx_HsHzD8w-VlEkK.DOHSQA.nLIPefFgcGHpFS_g3_yy1PM_S9E',
api_1          |  'HTTP_HOST': 'localhost:8080',
api_1          |  'HTTP_PRAGMA': 'no-cache',
api_1          |  'HTTP_UPGRADE_INSECURE_REQUESTS': '1',
api_1          |  'HTTP_USER_AGENT': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.75 Safari/537.36',
api_1          |  'PATH_INFO': '/dummy/b/count',
api_1          |  'QUERY_STRING': '',
api_1          |  'REMOTE_ADDR': '172.20.0.1',
api_1          |  'REMOTE_PORT': '46802',
api_1          |  'REQUEST_METHOD': 'GET',
api_1          |  'SCRIPT_NAME': '/api/tdp/db',
api_1          |  'SERVER_NAME': '00b34b55b068',
api_1          |  'SERVER_PORT': '80',
api_1          |  'SERVER_PROTOCOL': 'HTTP/1.1',
api_1          |  'SERVER_SOFTWARE': 'gevent/1.1 Python/2.7',
api_1          |  'werkzeug.proxy_fix.orig_http_host': 'localhost:8080',
api_1          |  'werkzeug.proxy_fix.orig_remote_addr': '172.20.0.1',
api_1          |  'werkzeug.proxy_fix.orig_wsgi_url_scheme': 'http',
api_1          |  'werkzeug.request': <Request 'http://localhost:8080/api/tdp/db/dummy/b/count' [GET]>,
api_1          |  'wsgi.errors': <open file '<stderr>', mode 'w' at 0x7f0f99d511e0>,
api_1          |  'wsgi.input': <gevent.pywsgi.Input object at 0x7f0f74e458d8>,
api_1          |  'wsgi.multiprocess': False,
api_1          |  'wsgi.multithread': False,
api_1          |  'wsgi.run_once': False,
api_1          |  'wsgi.url_scheme': 'http',
api_1          |  'wsgi.version': (1, 0)} failed with OperationalError
api_1          | 

• Static or animated images showing the UI behavior

Expected behavior

• Display the correct amount of the items.

in order to improve the user experience we could prefetch some views + their data if they are possible. Such that the data is loaded and the e.g. rendered in a dom node that is not yet attached to the real DOM.

or a way just to load the data of the view

such as view descriptions server db connector descriptions, ... first just highlight next provide for filtering and search

e.g. for performance testing

would be nice to have a CSV export of the sql view results

As a user, I want to get a feedback whether an action in the user interface has been (un)successful.

For example simple bootstrap alerts: see https://v4-alpha.getbootstrap.com/components/alerts/#dismissing

to better indicate embedded views and provide option to open in new tab

as used in lineup_dev standalone edition

currently you have to specify the idType (however already declared using .itemIDType and the range. However, creating the range is cumbersome. e.g. utilities to add/set/remove/toggle id/_ids would be useful

categories:

• information / table -> database info
• external page
• lineup
• vis
• others?

atm. there is this get/set Parameter construct with an onChange handler.

An alternative solution can be use to use an object for all params. Using a Proxy wrapper all changes can be tracked and e.g. trigger an event that view wrapper can use to track the parameter change.

e.g. for advanced server side processing as an alternative to the current JSON format

@sgratzl commented on Thu Nov 09 2017

in the next dev version we can switch to taggle since it is almost stable.

e.g. as an alternative for :ids for in db mappings and everywhere were mapping applies

e.g. whether the user has a certain role

.security(lambda user: user.has_role('role_name'))

when replaying actions can be compressed, e.g. certain views never shown since they will be replaced anyhow this includes executing all the parameter settings.

However, with the "useSession" feature in which previous filter settings are stored, these stored values will never be set and the initial parameters are not stored.

possible solution:

• when initializing a form builder element with a session value it should also trigger the change event which will trigger a recording of the parameter change

@lehnerchristian commented on Wed Aug 16 2017

As a user I want to filter a wide range of numeric values so that I don't have to select each element inside the range I want to filter

Currently the FormMap is only able to insert a single field (e.g. Select2). This works well for categorical columns, however for numeric columns each element would have to be selected separately (see screenshot)

It would be nicer to do something like:

@sgratzl commented on Thu Aug 17 2017

why isn't that an ordino issue?

for the deployment this should be a pure volatile redis database. If mixed the the default one used for mapping all the volatile cache content will be persisted, too.

see also https://github.com/phovea/phovea_data_redis/tree/redis_cache
and https://github.com/Caleydo/ordino/tree/cache

to locally use a different redis database.

in <workspace>/config.json

"phovea_data_redis": {
    "cache": {
      "hostname": "redis_cache",
      "timeout": 6000
    }
  }

6000 = 6000sec = 100min

in <workspace>/docker-compose.yml

  db_redis_cache:
    image: 'redis:3.2-alpine'
    command: 'redis-server --save "" --appendonly no'
 ...
 api:  
    depends_on:
      - db_redis_cache
    links:
      - 'db_redis_cache:redis_cache'

we could think about creating score templates = parameter filled up dialogs and provide them as "additional named scores"

@lehnerchristian commented on Mon Nov 27 2017

Feature Request

Short description

Overview (e.g. bar chart, chart js, boxplots, ...) + ranking (detail) view. Bi-directional filtering, synced selections, sidepanel should work for both. The Ranking view should only be shown when interacting with the overview (details on demand)

User stories

• As a developer, I want to have an overview linked with a ranking view so that I can create views which show overview first and details on demand
• As a biologist, I want to see a one view first without being overwhelmed of too many views

there seems to be a bug in the replaying of the taggle lineup sorting criteria -> a branch will be created for no reason

• link views by selection, i.e. the item selection of A is the input selection of B
• advanced layouting possibilities
• advanced linking - link only subsets of the shared elements with specific contained views

in the current version the named sets just store the set of rows that are listed.

In an advanced version, we could store the whole LineUp setup

• visible columns
• visible scores + parameters
• combined scores
• applied filters

Wouldn't that be the same as storing the entire session?

e.g. that a callback function is given with the signature

(engine) => list of dicts (that can be converted to json)

coming from Caleydo/ordino#22 point 4 and Caleydo/ordino#22 (comment):

When one column in LineUp and then trying to filter another column the second column should have the new domain as the boundaries (without the filtered rows)

E.g. when column B has a domain from [0, 10] and column A is filtered first, the remaining rows of column B could have the domain [1, 3]. when opening the filter dialog the domain of column B should be [1, 3] instead of the original [0, 10]

e.g. like an auto-fit option and zoom handles such that embedded views can scaled more easily

Currently based on select2 with some customizations. However the customizations should be unified and normalized. In different applications like dTiles the search box is already advanced

features

• multi select support (aka tags)
• ajax / data option provider
• custom formatting support
  • result vs selection
  • last search term used
  • access to underlying DOM element
• paste support. Text should be pasted and automatically parsed. Separators: \s;,.
  • split the input
  • validate the input individually and show current validation state (processing, error, success)
• drop file support - see paste support but just drop a text file into it instead
• grouping support: instead of selecting and representing a single item, items should be supported that represent a set of items of which all will automatically selected.
• custom data attributes in items. they need to have (id, text) but also others are optional and the objects shouldn't be modified
• easy way to get/set the currently selected items with AJAX support
• easy way to update the possible static options or change the AJAX data source.
• revise styles for e.g. invalid tags

the parameter is correctly set but the ui is not updated

the form builder is the only piece left in the core using d3. I order to avoid using d3(v3) in the core the FormBuilder needs to be refactored.

At the same time it can be improved.

• Atm there is no overview what are the options for specific form element descs. A builder pattern could help in this case.
• similarly when calling getParameter the view has to now which are valid parameter and what will be the return value. When using an interface object (IParams...) everything could be typed.
• select vs select2 both have different styles how to provide data, name/value vs id/text in which forms (data, promise, ajax, ... This should be refactored
• simplify select2 similarly to the search box in the Dashboard application, i.e. implementing a custom transport method
• provide option to extend a form desc e.g. with a visualization to generate the parameter like a histogram selector

@zichner commented on Mon Nov 06 2017

Bug Template

Release: dev

Steps to reproduce

1. Open list of all cell lines
2. Add two numeric columns (e.g., TPM and Raw Counts for gene TP53)
3. Drag and drop the 2nd score column in front of the first one (i.e., swap the order of the two newly added columns)
4. Make session persistent
5. Load persistent session

-> The column that got dragged and dropped doesn't contain data any more (all values are NaN)

PS: You can postpone this issue to a later milestone in case it is rather difficult to fix.

• Release number or git hash:
• Web browser version and OS:
• Environment (local or deployed):

Steps to reproduce

1. Open a list of genes
2. Select 2 genes (e.g. EPPIN and EVC2)
3. Open the Co-Expression plot
4. Select Gender for color coding
5. Hover over a mark and to see the name
6. Select this mark in the plot and choose the Database Info view
7. The names between the hover and the Database Info view are different (always CL-34)
8. Repeat steps 5. to 7.

Observed behavior

• the name inside the Database Info view is always CL-34

Expected behavior

• the name inside the Database Info view should be the same as when hovering the mark

while the parameter will be set through the provenance graph, it won't update.

see also #8 #19 #25

coming from phovea/phovea_core#125

since a (non-parameterized) score is essentially a vector we could wrap it to a phovea datastructure at least on the client side.

on the server side more information about the parameters are needed, see also #6

Currently the dialog is closed after a column was added
This would allow the user to add many columns without the need to click anywhere

we have multiple scores that take just one entity as input e..g a gene. Besides the data to show no additional parameters are needed. The idea is to unify those single scores such that you have only one unified dialog where you can select one or multiple data values that you want to add.

pro: database idempotent and this scores type can be used in other views, too

the view definition almost contains everything needed for generating a proper swagger.json file without any dynamic things.

missing

• arg typings (number, string, list of number, list of strings)

see Caleydo/lineupjs#418

additional event

• grouping criteria
• renderer type?
• ??

e.g. when dealing with Ensembl or EntrezIDs that still GeneSymbols are shown and selectable

in case of volatile id managements

• requires that e.g. the server is changed such that the real mapped ids will be stored
• client side changes such that the changed ids can be used
• adaption of the namedset4XXX filter logic
• mongo db update script??? to convert existing named sets

favicon + loading icon

such that we can avoid the Caleydo Icon

In case a score is restored using the score parameters stored in the provenance state, it would be useful to store a checksum that tells the user if the re-calculated scores are the same as the original ones. When the data in the database changes, this will not the case.

@zichner commented on Wed Sep 27 2017

Computed and uploaded columns can be removed, however, sometimes it might be desired to just hide a column and to be able to show it again later (without re-computation or re-uploading).

A search for HUNK1 should also show BRD4 with a note that it is a synonym of the searched entity

@sgratzl commented on Mon Nov 13 2017

currently empty

@thinkh commented on Wed Jan 10 2018

Steps to reproduce

1. Open gene list
2. Select a gene
3. Select Open Targets detail view

Observed behavior

The link is named //www.targetvalidation.org/

Expected behavior

• The link should be named either www.targetvalidation.org/... or https://www.targetvalidation.org/...

@thinkh commented on Wed Jan 10 2018

The same applies to other detail views, for instance Pub Med.

Select2 is limited as it does not support drop files and token validation.

Extract Select3 implementation from sgratzl/refactor into new branch.

• Update FormSelect2 to use Select3 options
• Update tdp_gene: search and validate

• Release number or git hash:
• Web browser version and OS:
• Environment (local or deployed): local

Steps to reproduce

1. Select a gene set
2. Select e.g. 2 genes
3. Open the Combined View
4. Choose a Data Subtype (columns will be added)
5. Persist session and open in another browser or private tab

Observed behavior

• In a persisted session: All steps are reproduced, except step 4 (no subtypes are chosen and therefore no columns)
• In a temporary session: the subtypes are preserved, therefore columns are added

Expected behavior

• The persisted session should behave like the temporary one and track the parameters of Select2

to improve the ATableView simple operations such as sorting would be useful.

loading icon isn't shown anymore