anucvml / vidat Goto Github PK

Dialog box for entering object information (classId, instanceId, colour, score and comment). List of objects in each frame.

It would be good also if a symbol appears on a corner of the bounding box for selecting that box and after selecting the box, the color of the box change, and then one can use "Delete" on the keyboard to delete the box or use arrow keys to move it. Also, for resizing, it would be good if the mouse cursor changes to some diagonal arrow symbol on the corners.

Video tutorials on how to use the tool for different annotation tasks.

It would be nice to get the option of aligning transcripts with the video. Something like this, please take a look at the video or demo, https://berndhuber.github.io/bscript/. So there is a synchronization between the transcript or script and the time in the video when the person says that word. It would be nice not just for tasks similar to the multi-modal opinion mining in video reviews but also video dense captioning.

Export keyframes as jpeg or png images so that they can be processed independently and to ensure timeframe and frame index align (for these frames) in other tools.

Don't create a new bounding box unless size is greater than some threshold, say 8 pixels in either direction. This protects against accidental clicks.

Cache greyscale images to allow for faster repainting during annotation.

I would like to have the annotations in a SQL database. Although, in the end, we can share JSON files with the community, I would prefer SQL for the trackability of the annotation process, scalability, simplicity, and order. sqlite3 it is a good and easy option.

I would like to import a list of keyframes from a file. Using one frame every second or so is just one alternative for keyframes selection, and it depends on what we want to do. For example, in the Spatio-temporal graph, I used Laplace variance for keyframe selection. Essentially, Keyframes needs to be flexible.

• Use video info panel to cache frames.
  • A video element to load the video and a canvas element to save the frames to Vuex.
  • Code can be reused.
• Delete the video in canvas panel, and re-layout.

Refactor javascript into modules. Report errors and show warnings as status text (flash messages). Warn when deleting. Warn when leaving page without saving.

stop -> back to where started

Should we use jQuery and bootstrap to simplify code, improve robustness and interface consistency? Or will this just add unnecessary complexity.

When browser zoom is not 100% bounding box resizing and moving seems to drift away from box boundary. Tested on Chrome under Windows.

I certainly believe that the current prototype of the annotation tool covers or will cover (after some iterations) everything that we need for doing the spatial, temporal, and natural-language annotation for the research on video analysis. However, I would like to point out some things about the use of this tool in the Amazon Mechanical Turk platform.

It is important to consider the annotation tool for two purposes. Of course, the annotation process itself and the evaluation of the annotators. Turkers want to get their payment as soon as they can. They want to receive their payment between two or three days. If we want to assess the annotations in time. We need a tool that facilitates the evaluation of annotations and annotators. It is for that reason that I believe this tool should have two purposes in mind.

• h :: help (this window): Will not be implemented
• p :: play video segment left panel only
• comma, < :: advance to next keyframe
• period, > :: retreat to previous keyframe
• [, { :: select left slider: Cannot be focused due to it is Vue component and focus is not implemented
• ], } :: select right slider: Cannot be focused due to it is Vue component and focus is not implemented
• left-arrow :: previous frame (when slider is selected)
• right-arrow :: next frame (when slider is selected)
• page-up :: jump to previous 10% of video (when slider is selected)
• page-down :: jump to next 10% of video (when slider is selected)
• plus (+) :: add a new segment
• delete :: delete currently active object
• shift :: duplicate currently active object when mouse down
• tab :: move to next field when editing objects

Allow temporal segments to take a class id in addition to free-text, allowing for action segment labels.

Relates to #9 (skeleton annotation and visualization).

• Make the video info panel more compact.
• Use dense mode in Quasar.
• Remove the video element or hide to cache frames.

According to the bounding box annotation, it would be good if the mouse cursor shape changes to + when it is ready to draw a bounding box.

Allow user to show and hide different annotation types (e.g., hide bounding boxes when labelling human pose).

Related to #20, At least preferences for showing/hiding GUI components should be be included in the first release (e.g., hide bounding box annotations if we're asking workers to label video segments).

Configure annotations such as object categories, colours, etc. Load from file. Configure user-preferences (e.g., keyboard mappings, defaults, etc) save in browser storage.

Allow fetching a remote video (possibly from YouTube). This could either be a GUI input option or encoded in the URL query parameters.

• Use QRange or QSlider.