iqtlabs / crviz Goto Github PK

Description

refreshing the dataset resets all group by and color by settings.

Environment

• 0.2.2
• all recent versions of Chrome and Firefox

Steps to reproduce

1. Load a dataset from a url
2. set the group by and color by to any desired values
3. click the refresh icon in the dataset panel

Expected results

the updated data should be displayed with the selected settings

Actual results

the updated data is displayed as if on initial load

Describe the feature you are requesting

implement a data strategy (probably a redux epic) to handle filtering datasets based on certain user defined criteria.

Other areas impacted

• Hierarchy config/selection
• Color By
• Search/Index

Lifecycle

Currrent:

Upload/Fetch->Load->Format->Index->Set Hierarchy->Display

New:

Upload/Fetch->Load->Format->Index->Filter->Set Hierarchy->Display

Links to

#112

Describe the feature you are requesting

Add unit tests covering some edge and error cases

Description

hostReportError.js:8 Uncaught TypeError: Cannot read property 'map' of undefined
    at A.Builder.<anonymous> (index-dataset-epic.js:80)
    at t (lunr.js:53)
    at e.v [as project] (index-dataset-epic.js:78)
    at e._next (map.js:75)
    at e.next (Subscriber.js:93)
    at t._subscribe (scalar.js:5)
    at t.subscribe (Observable.js:161)
    at t.call (map.js:51)
    at t.subscribe (Observable.js:158)
    at t.call (map.js:51)

Environment

• version of CRviz?

latest in master @c01c387eb1600023e7e619b2fe34ccd3e790a643

• which browser and its version?

chrome 65.0.3325.181

• versions of the following (if known and/or applicable)
  • node.js
  • RxJS
  • D3
  • Docker

Steps to reproduce

Run Poseidon and go to the instance of CRviz and select `Poseidon Network`, you'll see the above error in the Developer Console of Chrome

Expected results

What did you think would happen?

Not throw an error

Actual results

What really happened?

hostReportError.js:8 Uncaught TypeError: Cannot read property 'map' of undefined
    at A.Builder.<anonymous> (index-dataset-epic.js:80)
    at t (lunr.js:53)
    at e.v [as project] (index-dataset-epic.js:78)
    at e._next (map.js:75)
    at e.next (Subscriber.js:93)
    at t._subscribe (scalar.js:5)
    at t.subscribe (Observable.js:161)
    at t.call (map.js:51)
    at t.subscribe (Observable.js:158)
    at t.call (map.js:51)

Add docker build to travis config so that breaking changes to the docker build process are visible.

Describe the feature you are requesting

Add checks to enzyme tests to ensure that components receive correct css classes

Describe your use case

When automated PRs are created (by things like Greenkeeper and Snyk) it would be nice to have tests that fail due to meaningful changes in appearance.

we should implement UI level test scripts. We just had a bug that @lilchurro found when reviewing PR #73 where updating dependencies changed visual behavior. It was minor and easily fixed but we need to prevent this kind of thing in the future. It looks like the default for react is a combination of Jest and Enzyme but I'll dig into it more before I implement anything. It looks like storybook is another popular option.

Describe the feature you are requesting

Implement a way to display time series change with the context of an enclosure diagram.

Describe your use case

The visualization should:

• transcend groupings of data
• be applicable to multiple levels of grouping simultaneously
• be compatible with the anchoring methodology in order to keep visual reference consistent
• ideally be expansible to more than 2 groups of nodes, those the initial plan is to only work with two at a time

Dependent on

#108, #109

I was thinking we could probably optimize space and get a bit more room for our labels if we moved the circle text/labels UP, and also - if not too difficult - move the counts () underneath the label. (see my crude mockup, attached)

Describe the feature you are requesting

Remove the default to red for unknown nodes

Describe your use case

Red is typically used to denote a condition that is bad or requires remediation, unknown node does not necessarily fit this bill

Explain how this feature supports your use case? Other use cases?

As implemented, this is a hidden, undocumented behavior that only fits certain types of data within certain types of datasets. It is also the only scenario where anything is color coded by default, which is a sharp break with the way the rest of the application is designed.

Additional Info

N/A

It would be great if we could search on the data once it's loaded. Minimally I was thinking:

1. Initially scope it to the values of the existing JSON fields (e.g. "10.4.2.7") Let's start simple...
2. A search box in the UI so that one can do a basic string searches of the values
3. Some sort of visual queue that indicates whether results were found or not
4. Visually highlighting the results in some way
5. Some way of handling the situation where multiple results are found (e.g. Multiple hosts with the same name, or more generically, entities that have duplicate values)

Describe the feature you are requesting

Modify the data ingest to accept multiple datsets representing multiple points in time.

Areas Impacted

• Upload
• Download
• Fetch
• Format
• Hierarchy Config
• Index
• Search

I think the smartest thing to do here is to set the backend/data side up to accept an array of arbitrary length and then at the UI level restrict the components to looking at 2 points in the dataset. Probably just the first 2 at least to start, but there's no reason we couldn't give the users a way to say i want to see frame 1 compared to frame 17. Plus that leaves us in the position of not needing to upgrade the backend if we come up with a clever way to display things over time

Additionally the Epics will need to take in some sort of key designation for state storage so that we have an authoritative way to determine what objects at timestamp A correspond to at timestampB.

Links to

#109, #110

Steps to recreate:

1. load a dataset
2. set Color By to any value
3. remove the Color By by setting the dropdown back to '--'

The user will receive the following error in the javascript console:
TypeError: e.legend is undefined

Describe the feature you are requesting

When using the refresh functionality there is no indicator to the end user that it's doing anything.

Describe your use case

Clicking refresh button should ideally either have a spinner or some sort of 'working on it' or 'triggered' notification to the end-user so that they know it's doing something.

Explain how this feature supports your use case? Other use cases?

Without this, it's not obvious whether or not refresh actually did anything until it happens (assuming the refresh has new data).

Additional Info

if you search for say foo:bar and foo isn't a valid field, it will break any subsequent search queries

As it stands, many of our epics catch errors and display them to the user using window.alert this

1. is bad practice
2. is not very testable
3. leads to excessive code duplication

It would be better to use this logic to trap and transform any errors that we need to and then allow the results to bubble up to our components which should handle the error at the UI level.

Describe the feature you are requesting

Give users an ability to export the data and/or their view of the data

Describe your use case

In order to create shareability, but to also avoid the headaches of having to create a backend architecture to support shareable URLs, we should allow control state to be loaded from the uploaded data file. Additionally, this export must implicitly contain the data under view because the existing architecture fetches the data from a url so the only way to ensure that the user is sharing the exact data that the user is viewing is to package into one file.

Explain how this feature supports your use case? Other use cases?

This would also satisfy #50. It may be possible/make sense to:

• allow export of controls only as a mechanism for allowing users to "save" certain views or filters
• allow the export to point to a url so that users can pass around interesting views of live data

Additional Info

None

Links to

#50

Currently CRviz requires that within a JSON file upload the JSON is prefaced with:

{ "dataset": [

I believe this is required for some of the additional filtering/import options when bringing in data from an HTTP-based source. That said, it would be great if we could introduce some logic that could take a file with the appropriate "dataset" prefixing, but in the absence of that, just assume that the file is stock JSON and attempt to import it as such. (i.e. sans "dataset" reference)

This is specifically with a local file upload in CRViz - we probably want to keep the URL-based import as-is.

Describe the feature you are requesting

Modify the dataset select and upload UI to accept data in accordance with the specifications of #108 and to accept entry of a keying field for use in assigning object identity. It may be necessary to move some of the components into a modal dialog of some sort, or isolate certain UI elements into collapsible panels.

Links to

#108, #110

currently our dev and docker process involves a mix of npm and yarn. They both do the same general things and it is wasteful and confusing to use both. I think we need to pick one and go with it. My intention is to do some research before making a specific suggestion. Do any team members have a strong opinion one way or the other?

Implement a mechanism to allow users to fix the tooltip even after the mouse moves off of the tooltip's target node to allow better analysis and the ability to copy data from the tooltip.

currently, if there are updates to the file that is being fed by a URL selection, you have to refresh the whole page and load the URL in again to get the updated dataset, if there was a polling feature for this that updated the interface with any data updates on some interval, it would be useful for dashboard/HUD setups.

There's an issue importing CSV files when a quoted property in a CSV data object contains EOL characters. i.e.,

ID,name,description
123,Something,"This line\nhas a linebreak."

The linebreak will cause the CSV import to fail.

If you pass a single object instead of a dataset to CRViz instead of labelling the fields with values it will return indices.

Environment: https://cyberreboot.github.io/CRviz/
Steps to Recreate:

1. Select a dataset
2. Select a field to color by
3. Change the dataset selection

Expected Behavior: CRViz should switch to using the new dataset resetting the Color By field
Actual Results: the screen goes blank and an error can be seen in the javascript console
TypeError: e.legend is undefined

Describe the feature you are requesting

Write a quick start/FAQ type of guide for new users outlining the various features of CRviz and how to use them. Ideally it should be hosted on the web somewhere so that we can link to it both externally and from within the application. Maybe a markdown file in github so that we can version it alongside the application?

Describe the feature you are requesting

Instead of clicking a node to zoom to it use a standard set of zoom in/out buttons. This would give finer and more intuitive control over the level of zoom and free the mouse click gesture up to be used for some other feature.

Describe the feature you are requesting

Allow users to customize the colors when coloring by a specified attribute.

Describe your use case

When presenting data visualized in CRviz the arbitrarily assigned colors can inadvertently undercut the message you are trying to convey. For instance suppose a dataset has a property "Risk" with values low, medium, high, and critical. When coloring by this attribute it is possible for the colors to be assigned as

which is going to undercut the urgency generally associated with something considered a critical risk.

Explain how this feature supports your use case? Other use cases?

This would nicely complement the feature outlined in #103

clear search results in an error

RxJS 6 has discarded operator chaining in favor of operator piping
(for good reason see: https://github.com/ReactiveX/rxjs/blob/91088dae1df097be2370c73300ffa11b27fd0100/doc/pipeable-operators.md for more detail)
needless to say this presents several breaking changes for CRViz in that it breaks ALL of our epics. In the interests of smaller more performant js bundles and objects and eliminating unecessary dependencies we should upgade to RxJS 6.

Description

Describe your issue here.

Environment

• version of CRviz - 0.2.3.0 and v0.2.4.0-alpha
• Firefox and chrome

Steps to reproduce

1. load a dataset
2. color by a field
3. group by the same field that is colored by

Expected results

the same behavior as:

1. load a dataset
2. group by a field
3. color by the same field that is colored by
   i.e.:
   

Actual results

the grouping nodes turn black (or white if using the dark theme)

After the following sequence:

1. Select a dataset
2. Set a Group by field
3. Set Show nodes to false
4. Either change the dataset selection or remove all Group by fields

At this point, the canvas area will appear to be blank. What is actually happening is that there are only nodes to display, but CRViz has been instructed not to show those. This will be confusing to some users, we should probably reset the value of Show nodes to true on these events in order to avoid confusion

Describe the feature you are requesting

Create UI components to allow users to create, apply, and remove filters

Linked to

#111

Describe the feature you are requesting

Allow a user to add authorization credentials when selecting Custom URL as a datasource
perhaps by entering:

• auth token
• username/password

when searching using the notation "fieldaName: value" the capitalization of the fieldName must match exactly or the search will fail.

Indexing a dataset is inherently a relatively intensive process. There is no reason this can't be done in the background while the user is otherwise able to use and interact with the application. This operation should be implemented as an RxJS epic so that if a new search library is introduced then there is a single point to insert appropriate index creation logic and if a search index is not used there is no need to dispatch the action.

Description

Describe your issue here.

Environment

• 0.2.3 branch 109-tsu-ui
• Firefox and Chrome
• versions of the following (if known and/or applicable)
  • N/A

Steps to reproduce

1. select a dataset
2. click the "minus" icon to collapse the dataset panel
3. click the plus icon to reopen the dataset panel

Expected results

that the previously selected dataset information would be populated

Actual results

the dataset panel looked the same as it does on initial load.

this would be particularly useful when using a URL where you may not already have the JSON file of what you're viewing

Describe the feature you are requesting

Allow users to add a layer of markup on top of the displayed visualization.

Describe your use case

When presenting data visualized by CRviz to other audiences it would be exceptionally useful to have some ability to highlight or otherwise direct attention to some particular item or group of nodes.

Explain how this feature supports your use case? Other use cases?

Additionally this feature could allow users to display supplemental or contextual information. Ideally there should be a mechanism to export the markup as an image, possibly without displaying the controls.

Describe the feature you are requesting

Calculate and display statistics on the properties of the dataset being displayed by CRviz, such as

• min
• max
• average

Design Questions

1. What statistics do we want to calculate and what datatypes are they valid for? We probably need to generate some sort of matrix with this. I'm thinking about the fact that min, max, and average make sense for integers, but if you are looking at dates average is useless but min and max can be very meaningful.
2. How do we display/convey this data to the user?
3. How does this interact with other existing/planned features such as filtering and markup?

We really should display version info somewhere in the application. We have it up on github.io and we want outside users to report issues against it then we really should provide a mechanism for everyone to know what version of the software is being used.

Describe the feature you are requesting

Add google analytics to gh-pages branch

Description

because of an id conflict in the upload control, the htmlFor would always find the first instance of a fileUpload element in the DOM tree, leading to confusing behavior from this in the upload event

Environment

• 0.2.3

Steps to reproduce

1. Click on the file upload button in the second instance of the file upload control
2. select a file to upload

Expected results

the filename would populate in the upload area of the second dataset control.

Actual results

the filename would always populate in the upload area of the first dataset control.

Description

Errors are frequently reported using window.alert("message"). This is bad practice and difficult to test.

Environment

• version of CRviz?
  0.2.3
• which browser and its version?
  all browsers/versions
• versions of the following (if known and/or applicable)
  N/A

Steps to reproduce

N/A

Expected results

Errors should be collected and displayed in a fashion that is:

1. consistent across browsers
2. controlled by our dev team
3. testable
4. customizable

Actual results

window.alert was used

Found in version: 0.2.1

Steps to reproduce:
run the command: docker build -t crviz-gh -f Dockerfile.gh .

Expected result
the docker image builds successfully

Actual result:
received the error: /bin/sh: git: not found
The command '/bin/sh -c git config --global user.email " "' returned a non-zero code: 127

It would be nice if the URL add breadcrumbs when elements are changed in the control panel, so that someone could use the URL to get all of their particular settings back without having to click through all of them in the interface every time.

Describe the feature you are requesting

A button or function to reset the display clearing anything like coloring, hierarchy selection, zoom, etc. with the exception of the loaded dataset since the user might be pulling from a remote url and will not want the underlying data reset.

tep 6/11 : RUN yarn install ---> Running in 7368cd02a803 yarn install v1.7.0 [1/4] Resolving packages... (node:6) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead. warning [email protected]: breaking changes in minor version [2/4] Fetching packages... info [email protected]: The platform "linux" is incompatible with this module. info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation. warning [email protected]: The engine "gitbook" appears to be invalid. [3/4] Linking dependencies... warning " > @fortawesome/[email protected]" has unmet peer dependency "prop-types@^15.5.10". warning "react-scripts > babel-preset-react-app > babel-plugin-transform-dynamic-import > @babel/[email protected]" has incorrect peer dependency "@babel/[email protected]". [4/4] Building fresh packages... success Saved lockfile. Done in 29.97s.
AND
Step 8/11 : RUN npm install -g serve ---> Running in f93ccc900906 npm WARN notice [SECURITY] lodash has the following vulnerability: 1 low. Go here for more details: https://nodesecurity.io/advisories?search=lodash&version=2.4.2 - Run npm i npm@latest -gto upgrade your npm version, and thennpm audit` to get more info.
/usr/local/bin/serve -> /usr/local/lib/node_modules/serve/bin/serve.js

• [email protected]
  added 41 packages in 2.115s`

Allow user to upload a CSV file; convert it to json format and feed it to CRviz.