I'm thinking about a find and replace feature and I'd like your input.

Suggested behaviour

On selecting Find in the menu or using a shortcut, display a panel at the bottom of the screen:

Users can press the buttons, or in some cases a keyboard shortcut, to:

• find the value within the current column or across the table
• find the next or previous value
• replace the value with another
• replace all the values with another

Other Ideas

• Implement the solution in the Toolbar

What do you think @albreez @abhiravbansal @TacoSandwich

Needs to work across macOS, Windows 7, 8, 10, Linux

Expected Behaviour

I am able to add a regular expression to a pattern constraint

Current Behaviour

Pattern constraint not shown in Column Properties panel

Steps to Reproduce

1. Open Data Curator using the icon
2. Click column properties
3. Note no Pattern constraint

Your Environment

• Data Curator version 0.3.0
• Operating System macOS Sierra 10.12.2

Expected Behaviour (for problems)

When using the menu command:
https://relishapp.com/odi-australia/data-curator/docs/edit/select-all
When using the menu command, I expected it to select everything in the table as it does when using the ctrl+a keyboard shortcut.

Current Behaviour

When using the menu command, select all literally selects EVERYTHING including menu labels and the tab names.
The ctrl+a keyboard shortcut works as expected.

Your Environment

• Data Curator version 0.2.3
• Windows 7

Menu items should be updated to reflect specification...

Expected Behaviour

https://relishapp.com/odi-australia/data-curator/docs/file/new

Current Behaviour

The new tab is created but the cursor is not placed in the first row and column

Steps to Reproduce

1. Select New from the menu
2. Cursor not in first row and column

Your Environment

• Data Curator 0.2.3
• Operating System macOS Seirra 10.12.6

If ragged rows are checked on opening the file, and the only choices are:

• fix ragged rows - setting the number of columns to the maximum number of columns
• setting the number of columns to the number of columns in the first row

Then is there any need to have Fix Ragged Rows in the menu?

See https://discuss.okfn.org/t/is-there-a-machine-readable-list-of-open-licences-and-urls/5503
and okfn/opendefinition#7

See also http://opendefinition.org/licenses/ perhaps limit drop down list to Recommended conformant licenses

Expected Behaviour

https://app.cucumber.pro/projects/data-curator/documents/branch/develop/test/features/file/close-tab.feature

Current Behaviour

Nothing happens when using the Close Tab menu item or keyboard shortcut

Steps to Reproduce (for problems)

1. Open two data tabs
2. Use the Close Tab menu item or keyboard shortcut
3. Nothing happens

Your Environment

• Data Curator 0.2.3
• Operating System macOS Seirra 10.12.6

This issue was discovered when I tried testing DC using a table containing 1000 rows to see how it would handle. There are a two issues here;

1. The scrollbar behaviour is not as expected, especially the horizontal scrollbar
2. Scrolling of the table vs scrolling of the application and the visibility of the menus et. al.

Desired Behaviour

If the number of columns in the table exceed the number of columns that can be displayed on the screen the horizontal scrollbar should display at the bottom of the screen, irrespective of the number of rows loaded.
When user clicks on the horizontal scrollbar at the bottom of the screen it should allow them to move it sideways to scroll through the columns offscreen.
When user clicks on the vertical scrollbar at the right of the screen it should allow them to move it up and down to scroll through rows offscreen.
Additional rows should load as the user scrolls down until the last row at the bottom of the table is reached.
The top menu (Validate | Column | Table etc.), tabs, column reference (A,B,C,D etc.) and column headings should all remain visible as the user scrolls down.
To save data and load time, DC should only display a set number of rows by default (which it appears to do). However, there needs to be an option that allows the user to increase the maximum number of rows to load, including loading all rows.
There also needs to be the ability to pin and unpin the top menu and tabs to the top of the screen so that the user has the ability to control whether they remain visible or be hidden as they scroll.

Current Behaviour

When a large table is loaded that contains more than the maximum number of rows visible on the screen DC only loads the rows that can be displayed. This is fine, but when the user scrolls down and clicks on the horizontal scrollbar to scroll sideways, DC loads additional rows at which point the horizontal scrollbar (which is supposed to allow the display of additional columns, not rows) disappears.
This means the user cannot use the horizontal scrollbar to navigate sideways until all rows in the table are loaded. If there are >100 rows in the table, this will become a problem.
The top menu (Validate | Column | Table etc.), tabs, column reference (A,B,C,D etc.) and column headings also disappear from view if the scrollbars are used.
There is no easy way to navigate to the last row at the very bottom of the table.

Steps to Reproduce

1. Open Data Curator using the icon
2. Open a new file (any type) with >100 rows
3. Start scrolling

Your Environment

Data Curator version 0.2.3
Windows 7

Images

Expected Behaviour

Across the application Dialogs should be consistently styled.
Correct style:

Current Behaviour

Open file with ragged rows

Open Excel Sheet

Your Environment

• Data Curator: 0.2.3
• Operating System macOS Seirra 10.12.6

Building an App is one thing. Sustaining it over time is another.

Should we, at an appropriate point in time, request (or say how to make) contributions - in kind or cash.

E.g. https://github.com/cyclejs/cyclejs/blob/master/CORE_TEAM.md

Desired Behaviour (for feature requests only)

Assuming the application icon is meant to display.

Expected Behaviour (for problems)

as above

Current Behaviour

Currently the icon displays as broken.

Steps to Reproduce

Following the testing steps outlined here https://relishapp.com/odi-australia/data-curator/docs/data-curator/about
I,

1. Open Data Curator using the icon
2. Click on Data Curator from the Application menu
3. Select About Data Curator

Your Environment

Windows 7
Data Curator version 0.2.3

Additional note - when following the Gherkin steps on Relish, the procedure is inconsistent with the application. For example;

Given I have opened Data Curator
When I select "About" from the menu > There is no About in the application menu. The menu item where this item is found is Data Curator (see image)
Therefore; the Gherkin script should read something like (please forgive my syntax);
Given I have opened Data Curator
When I select "Data Curator" from the application menu
And I select "About Data Curator" from the "Data Curator" menu
Then... [script continues]

Please disregard this if the application is changing to comply with the original script.

Expected Behaviour

https://relishapp.com/odi-australia/data-curator/docs/help/initiate-an-edit-feature-using-a-keyboard-shortcut

Current Behaviour (for problems)

Steps to Reproduce (for problems)

1. Select Keyboard Shortcuts menu item

Your Environment

• Data Curator: 0.2.3
• Operating System: Windows 7

Expected Behaviour

https://github.com/ODIQueensland/data-curator/blob/master/test/features/data_curator/quit.feature

Current Behaviour

Work is saved but prompted to save work on Quit

Steps to Reproduce

1. Create data
2. Save data
3. Quit
4. Prompt presented to save unsaved work, when application should quit

Your Environment

• Data Curator 0.2.3
• Operating System macOS Seirra 10.12.6

adding a relish push to continuous integration would mean that features published to Relish would always be up to date

gem install relish
relish push <path>

What's happening with Data Curator?

We'll occasionally post news here so you can follow our progress without needing to watch this repository and see every conversation.

Simply press the Subscribe button on this Issue to follow along.

See what we've done in the Changelog. You can also see what we have planned for each milestone.

Subscribe to our Atom feed to be notified about our releases.

This conversation is locked so only collaborators on this repository can comment here. That means you won't get spammed with notifications - unless we're making awesome progress ;-)

Thanks to @sjackman for suggesting this idea.

In Data Curator version 1, we'll deliver a set of core features.

I'm starting to think about version 2. I'd love to hear your ideas.

Here's mine...

1. Tools to help package and publish data

1.1 Set Preferences

Set preferences for default values and behaviours to speed up packaging of data.

1.2 Publish data package to CKAN

Using securely stored API credentials enable publishing the data package to CKAN

1.3 Import Column Properties

Access a library of pre-defined column properties and import column properties to apply consistent descriptions and constraints to similar data

1.4 Print

Print sample data and metadata to show to Data Custodian as part of the open data release approval process

1.5 Open data from other sources

Open files directly from the web instead of downloading them to your computer.

• Open from csv file stored on web
• Open from Google sheet
• Open a file using Drag and drop

1.6 Support Custom CSV Dialects

Sometimes publishers need to open and save CSV files using Custom CSV Dialects

2. Tools to help correct data

2.1 Create constraint from column

To create an enumeration constraint based on the data in the column:

• read the data in a column
• determine the set of unique values
• automatically generate an enum constraint in the column properties

2.2 Create look-up table from column

Use the set of unique column values to create a look-up table and a foreign key relationship between tables.

2.3 Validate foreign keys against data at a url

Validate data against a shared code-list stored at a url. This allows organisations to use a shared code-list to lookup valid values.

This is dependent on an anticipated change to the Frictionless Data specification

2.4 Find, Replace, Sort and Filter

• Find and Replace within a Table
• Find and Replace within a Column
• Sort Columns
• Filter Column

2.5 Validate as you type

Validate data as you type data in a cell, check it matches the Type, Format and Constraints

3. Tools to data consumer understand and view the data

3.1 Generate data quality information and report

Read the data to generate data quality statistics. Include the report in the data package.

3.2 Define a default visualisation

Use an emerging specification to set a default view for a data package

3.3 Read-only mode

Prevent unintended changes being made to the data and metadata

4. Improve application reach and support

4.1 Check for Updates

Automatically check for application updates

4.2 Configuration file

The collection of column, table and data package properties will be driven by a configuration file. This file will enable:

• new properties to easily added to the application as standards change
• setting default values across an organisation enabling consistent publishing
• supporting other metadata standards e.g. W3C CSV for the Web

4.3 Improve Help, Support and Onboarding

• provide link to external help website and update content to match application features
• embed help in app so internet access is not needed
• provide onboarding tour of key features when user opens the app for the first time

4.4 Publish to App Store and provide ongoing support

• Provide confidence and ease to install application
• Fix bugs quickly

Desired Behaviour (for feature requests only)

Suggestion only - Give each new tab a unique name e.g. Untitled.csv then Untitled2.csv then Untitled3.csv by default
This is to avoid errors and possible confusion later on.

Current Behaviour

Each time a new tab is opened it is given the same name "Untitled.csv" by default.

Steps to Reproduce

Applies to all 3 methods of opening a new data tab

1. Select File > Select new
   or 2. Press Ctrl + N
   or 3. Click on the + next to the open tabs

Your Environment

Data Curator version 0.2.3
Windows 7

Expected Behaviour (for problems)

Cut: https://relishapp.com/odi-australia/data-curator/docs/edit/cut
Copy: https://relishapp.com/odi-australia/data-curator/docs/edit/copy
Paste: https://relishapp.com/odi-australia/data-curator/docs/edit/paste

Current Behaviour

Functions do not work when using the menu commands.
Keyboard shortcut commands work.

Environment

• Data Curator version 0.2.3
• Windows 7

Expected Behaviour (for problems)

Data rows in the table should align with the row reference grid

Current Behaviour

See image - the rows do not align with the grid (the blue line showing the selected row isn't straight)

Steps to Reproduce

1. Open Data Curator using the icon
2. Open a data file (e.g. .csv)
3. Select a row and see if it is properly aligned with the grid.

Your Environment

Data Curator version 0.2.3
Windows 7

After initial merge from Comma Chameleon, there are still a couple of references that need to be removed. e.g. Menu Item names

https://github.com/ODIQueensland/data-curator/search?utf8=%E2%9C%93&q=Chameleon&type=

Specifically:

• this lizard image from the css background
• About and Hide Comma Chameleon menu items

Use of semantic versioning requires "Software using Semantic Versioning MUST declare a public API."

Do we need to adjust changelog.md?

Rather than selecting Help Data Curator from the Help menu and launching a browser with a URL to an external help site, provide in-app help.

This help request is just to provide some sample code to implement help tags. Project maintainers will provide the help text and apply across the application.

Desired Behaviour

https://relishapp.com/odi-australia/data-curator/docs/other/show-help-tags

Inspired by https://developer.apple.com/macos/human-interface-guidelines/user-interaction/help/

Add acceptance tests

• release 0.1.0
• release 0.2.0
• release 0.3.0
• release 0.4.0
• release 0.5.0
• Backlog

http://keepachangelog.com/en/1.0.0/

Panels should be a fixed percentage of the screen width and not drop in a responsive menu button at small sizes.

Drop button labels as width reduces.

Take styling clues and behaviours from Slack App (mac OS)

Expected Behaviour

https://app.cucumber.pro/projects/data-curator/documents/branch/develop/test/features/file/close-tab.feature

Current Behaviour

Close Tab closes the tab without warning that the work is unsaved

Steps to Reproduce (for problems)

1. Enter some data
2. Press the close button on the active tab
3. Tab with unsaved work is closed

Your Environment

• Data Curator 0.2.3
• Operating System macOS Seirra 10.12.6

Expected Behaviour

https://relishapp.com/odi-australia/data-curator/docs/file/open-a-comma-separated-value-file

Current Behaviour

All scenarios test successfully however information messages are output at the bottom of the window

After selecting OK...

Steps to Reproduce (for problems)

1. Open a csv with ragged rows
2. Select OK to correct them
3. Messages appear

Your Environment

• Data Curator: 0.2.3
• Operating System macOS Seirra 10.12.6

Expected Behaviour

When I open an Excel sheet with a graph and no values in any cell, display an empty table

(No Acceptance Test written for the scenario)

Current Behaviour (for problems)

Steps to Reproduce

1. Create data in Excel
2. Create a graph from the data in a second tab
3. Save Excel
4. Open the Sheet with the graph in Data Curator

Your Environment

• Data Curator: 0.2.3
• Operating System macOS Seirra 10.12.6

Desired Behaviour

Drag a file on to Data Curator to open it

Acceptance test

Resources

• https://electron.atom.io/docs/api/file-object/
• https://ourcodeworld.com/articles/read/288/how-to-handle-drag-and-drop-file-feature-in-electron-framework
• https://gist.github.com/dcposch/1e546279f162de7d644b224058b27fb0#file-electron-drag-drop
• use CSV sniffer to determine dialect https://www.npmjs.com/package/csv-sniffer

Desired Behaviour

Either remove Toggle Developer Tools from menu and replace with a keyboard shortcut
or
Only show Toggle Developer Tools in the menu when in Development environment

See https://electron.atom.io/docs/api/accelerator/

Expected Behaviour

When a save a file, the path to that file should not become the Application Name

Current Behaviour

Steps to Reproduce (for problems)

1. Create some data
2. Save it
3. See the Application Name change

Your Environment

• Data Curator: 0.2.3
• Operating System macOS Seirra 10.12.6

Unintended consequence from Save / Save As

I clicked on File > Open Excel Sheet
The program opened file explorer.
I selected the file I wanted to open (it was a .csv)
I was presented with a blank dialogue with no error message

Expected Behaviour

The file should either open or I get an error message

Current Behaviour (for problems)

See above

Steps to Reproduce (for problems)

Provide a link to a live example, or an unambiguous set of steps to reproduce this bug. Include code to reproduce, if relevant.

1. I clicked on File > Open Excel Sheet
2. The program opened file explorer.
3. I selected the file I wanted to open (it was a .csv)
4. I was presented with a blank dialogue with no error message

Your Environment

Windows 7, version 0.2.3

@Stephen-Gates @mattRedBox

https://github.com/theodi/toolbox/wiki/Labels-In-Use-Across-the-ODI-Toolbox

Wait for https://github.com/theodi/toolbox/blob/master/.github/labels.json to be updated to import new labels

Here's a idea that is perfect for first time contributors with javascript skills...

Desired Behaviour

The usual "Open Recent" menu item behaviour...
https://app.cucumber.pro/projects/data-curator/documents/branch/develop/test/features/file/open-recent.feature

There already a placeholder menu item in the code.

There some implementation tips in:

• https://electronjs.org/docs/tutorial/recent-documents
• a tutorial with answers

Expected Behaviour

https://relishapp.com/odi-australia/data-curator/docs/file/save

Current Behaviour (for problems)

First time Save doesn't prompt for filename and locations

Steps to Reproduce (for problems)

1. Enter data
2. Select Save menu item
3. No prompt presented
4. Unclear if file was saved and if so, where

Your Environment

• Data Curator 0.2.3
• Operating System macOS Seirra 10.12.6

The toolbar includes backlog items - Find and Replace and Publish

Data-curator should be Data Curator

Toolbar should not have a border line and white background should match toolbar colour. e.g.

Expected Behaviour

https://relishapp.com/odi-australia/data-curator/docs/edit/redo

Current Behaviour

After insert row above, when I invoke redo using menu item or shortcut, the action isn't redone.

Your Environment

• Data Curator: 0.2.3
• Operating System macOS Seirra 10.12.6

See Acceptance Test

Indicate that Provenance Information will be stored using the CommonMark version of Markdown http://commonmark.org

Add logo to readme https://github.com/dcurtis/markdown-mark

Explore libraries to implement:

• https://github.com/markdown-it/markdown-it (dated but feature rich - used in discourse)
• https://github.com/jgm/CommonMark

UI should be something similar to adding an Issue in GitHub

Expected Behaviour (for problems)

When I delete all rows and columns, leave a 1x1 table in place

Current Behaviour

You can delete all rows and columns leaving you with

Steps to Reproduce

1. Open Data Curator using the icon
2. Place the cursor in Row 1
3. Right click, delete row

Your Environment

• Data Curator version 0.2.3
• Operating System macOS Sierra 10.12.2

Display number of release downloads (latest or all) by platform so the growth of Data Curator can be demonstrated over time.

see

• theodi/comma-chameleon#166
• http://metrics.theodi.org/metrics/comma-chameleon-downloads/2017-09-08T19:47:41+00:00

If a movement causes the cursor to move off the screen but it is still within the table, then scroll the table so the cursor remains visible on the screen. (OCC-250)

Expected Behaviour

https://relishapp.com/odi-australia/data-curator/docs/other/move-cursor-in-table

Current Behaviour (for problems)

When cursor goes off screen on Move Up/Down, the table doesn't scroll to keep cursor on screen.
Move Left/Right performs the correct behaviour

Steps to Reproduce (for problems)

1. Add rows that exceed the screen size
2. Use arrows to move up / down
3. When cursor goes off screen, table doesn't scroll to keep cursor on screen

Your Environment

• Data Curator 2.3
• macOS

Current Behaviour

The Tab close button is a bit loud.

Also Tab baseline should match panel header baseline

Expected Behaviour

• change colour of X to grey
• remove background colour (red)
• show X on hover over Tab
• hide X when no hover over Tab
• shade inactive Tabs a darker colour than the active Tab (e.g. the current hover over colour)
• baselines align e.g.

Expected Behaviour

The scroll bars should focus on:

• the table (horizontal and vertical scroll bars)
• the panel (vertical only as panel resizing limited to a useful range e.g. 20%-40% screen width)

Footers should be fixed on screen

For example (in Numbers):

Current Behaviour (for problems)

• Scroll bar applies to whole screen
• Toolbar should not scroll

The contributing.md file states that you've adopted ODI Toolbox labelling. This hasn't been implemented.

See also theodi/toolbox#15

Please consider supporting reading and writing tab delimited (TSV) files.

A question I was asked (and a feature that exists in Comma Chameleon)...

Why doesn't the planned features include publishing?

Publishing a data package to an open data portal is the next step in the data packaging workflow. Surely you could add a button to send the datapackage.zip to a your CKAN open data portal, Datahub.io or other portal.

When speaking with potential users of Data Curator, they expressed concerns about accidentally publishing open data before approval has been granted by the Data Custodian. They said they would present sample data, the schema and provenance information as part of the approval process.

So it's on the backlog for now.

Notes:

• Can the CKAN API accept Data Packages, like it accepts them through the user interface. Perhaps some other tool would be helpful.
• The Octopub API does not currently accept Data Packages.

Apply similar styling to right panel close button as suggested in #35