Currently the user is able to draw an annotation only once.

This includes:

• Slider for varying brightness

• Slider for varying contrast

• Automatic intensity adjustment

• Select different color for each channel in the image

• Toggle/untoggle different channels in color space

User should be allowed to:

• Change the name of a category
• Change the assigned color of a category

Currently the stage (and the image in it) are located in the top left corner of the space between the two side bars. We want it to be centered horizontally.

It shouldn't be centered vertically, instead we are thinking of a bit of padding at the top.

The image inside the image container should be centered.

It would be nice to have the side bars animated. For example when the user clicks on an new tool, the right side bar with the different options will slide in.

For each selected pixel, we would like to get the corresponding mask which will be used for storage and export purposes. Currently we have chosen our mask to be a dataURL, but we might change to a more compressed format later.
Here I am listing the tools that have been integrated in our ImageViewer app, but this is a non-exhaustive list which will be changed as we incorporate more tools:

• Rectangular Selection
• Elliptical Selection
• Lasso Selection
• Polygonal Selection
• Magnetic Selection
• Object Selection

In the lasso selection tool, if the user clicks (without releasing), drags for drawing a path, and releases on the starting point, the resulting lasso selection immediately disappears.

When resizing our annotation tool window, the resulting position is not mapped to the current location on the canvas. As a result, annotations are drawn with an offset.

In principe this should have been solved with our call to transform.point(), but it seems like a bug may have been introduced somewhere.

When a lasso or polygonal selection has been drawn and selected, clicking on the resulting instance should make the transformer appear. Currently, clicking results new polygon annotation being drawn.

These bugs include:

• The shape does not close once clicked on initial anchor point.
• The control anchor points do not show.

This should be done to all of our annotation components that include a sequence of strokes.

• Rectangular, elliptical, lasso, and polygonal selection
• Magnetic selection
• Quick selection
• Color selection
• Object selection

The app would be more attractive when first loaded if we:

• Show a microscopy image instead of the current dog image, and

• Pre-annotate this image with good looking annotations and colors. Corresponding categories will have to be pre-made (e.g., nuclei).

ObjectSelection currently has two problems:

• The contour map has unwanted small holes. We'd like to do some processing on that to get a clean contour.
• The coordinates that we have for the contour map are unordered. This results in lines that intersect and overlap each other to create a mess and cover the whole object.

When our path has been closed, we don't want to see an initial anchor point and the last control anchor point.

We are currently updating the countour and mask props for the resulting selection, but not boundingBox.

From the following fields:

• Biology (e.g., histology and pathology)
• Medicine
• Astronomy

Found by @DavidStirling : it is possible to resize rectangles and ellipses until they're outside the canvas, after which they can't be recovered.

• On primary click should flood the image using the mouse position as the starting flood position
• Flood tolerance should be configurable (i.e. the component should have a tolerance: number prop)

Note that the algorithm was already written in flood.ts.

Currently when the user clicks on a selection to edits its category, all of the other selections are changed to that category as well.

As a result, the JSON export does not have the right category in it.

• Wheel in-out

• Rectangular Selection

• Click in-out

• Reset to original image scale

• Better typography for "Zoom in-Zoom out"

• Dash stroke should not become more dense as zoomed in

• "Automatically center" functionality

Suppose we have an instance on our image. Clicking on it currently results in the onMouseDown() event of the selected operator to be called. (as opposed to show show the transformer around it).

This is why when clicking on an instance in Lasso or Polygonal mode, we involuntarily enter the drawing mode. (It actually happens if we are in the Rectangular/Ellipse tools, but we don't see it because we click and release immediately instead of dragging to see the shape being drawn.)

@0x00b1 When the user clicks on the Stage, should we be checking whether the clicked (x, y) coordinates belong to one of the bounding boxes of our existing annotations? If yes, we should do an early return before entering the operator's onMouseDown?

For the mask, we'll want to draw circles and use the this.points as our circle origins. The circle radius is determined by the selected brush size. See https://github.com/dulnan/lazy-brush for inspiration.

The contours will be then obtained from the mask.

Currently the drawing of all of the tools in ImageViewer's default is very laggy. We suspect that it might be due to a new class of the tool being instantiated, with each new render.

When exporting, the category that was first selected when doing the selection is the one that is used by the export file. The category for that annotation should be the one that was last assigned (not the one at the time of the selection).

Each annotations should be exported as a JSON object and saved to a file.

Make a polygonal selection and either Esc or Enter --> the selection disappears but if you start a new selection the previous starting anchor appears.

The user should able to select an existing instance and delete it with backspace or escape button.

Stage has become dense with all of our `useEffect, let's try to group some of these into custom hooks, when appropriate.

• Add a prop that allows user to select either rectangular or lasso selection.
• Implement the rectangular and lasso selection functionality from the rectangular component and the lasso component
• Crop the selection made with either the rectangular or lasso tools
• Use TensorFlow.js to create a saliency map for the cropped region
  • the saliency map should be a binary mask with exactly one object
  • see example on GitHub
• Create a boundary of the positive values of the saliency map and display this as the selection

Currently when clicking on an instance to select it, we enter the onMouseDown method of that tool and start drawing with that tool. It is currently not possible for the user to click on one of these existing instances without immediately entering the drawing mode.

Add in the following visual cues for Add, Subtract, or Intersection modes:

• On mouse down:

  • Cursor changes
  • Anchor box points of first selection are removed

• On mouse move:

  • Lighter line opacity for any contour that will disappear

• On mouse up:

  • Bring anchor points back
  • Marching ants

There is a bug that happens somewhere in the encoding and decoding of a mask created by PenSelection.
The top image shows the image that goes into the encoding function. The bottom image shows the image that comes out of it.

This behavior is particularly clear when selecting the "Microscopy" image in our sample images. I have been able to replicate it sometimes with the "dog" picture.

Clicking and dragging to draw the tolerance map will show the resulting map in the very left side of the image instead of on the pixel location of our cursor. When releasing and pressing enter, though, the mask at the right location shows.

Implement the following:

• Create a superpixel representation of the image (i.e. using a method like quickshift, k-means, or even compacted watershed)
• onStart event, create a point and display the corresponding superpixel (e.g. as a yellow polygon with 50% opacity)
• onMove event, create a point for each onMove position (i.e. like lasso selection) and display the corresponding superpixels (i.e. the superpixels that contain each point)
• add a size prop that determines the relative size of the superpixels