Giter VIP home page Giter VIP logo

beaverdam's Introduction

BeaverDam

Video annotation tool for deep learning training labels

About

This tool is for drawing object bounding boxes in videos. It also includes support for Amazon Mechanical Turk. See the paper.

With small amount of changes, you can also:

  • Draw bounding boxes in images
  • Add additional attributes in bounding boxes
  • Use a custom keyframe scheduler instead of user-scheduled keyframes

This tool currently does not support semantic segmentation.

Installation

  1. Clone this repository.
  2. cd BeaverDam
  3. Make sure Python 3 is installed.
    If not: brew install python3 (Mac) or sudo apt-get install python3 (Ubuntu)
  4. Make sure virtualenv is installed.
    If not: pip3 install virtualenv or maybe sudo pip3 install virtualenv
  5. Make the Python virtualenv for this project:
    scripts/setup
  6. Download sample data:
    scripts/seed -f

When running any ./manage.py commands, use source venv/bin/activate to enter venv first.

See /deployment for tips on using BeaverDam for production.

If using mturk

Replace the credentials below with your own:

export AWS_ID="AKIAAAAYOURIDHERE"
export AWS_KEY="YOURmturkKEYhere5DyUrkm/81SRSMG+5174"

When ready for real turkers, edit MTURK_SANDBOX to False in settings.py.

It is recommended to use IAM keys with only mturk permissions instead of root key.

Running the server

scripts/serve

Then navigate to localhost:5000 in your browser.

Need to run on a custom port? env PORT=1234 scripts/serve

For actual production deployment, we recommend using standard Django deployment procedures. Sample scripts using uWSGI & nginx are provided in /deployment. Remember to set DEBUG=False in settings.py.

Making accounts

Login is required to authenticate any changes. Turkers do not require accounts and are authenticated by BeaverDam via Mechanical Turk.

To make a superuser account, run inside venv ./manage.py createsuperuser If you are using sample data, login with username test and password password. Additional non-turker worker accounts can be created via /admin.

Videos

To add videos, one must upload the video to a CDN (or use /annotator/static/videos to serve on the same server), then create a Django video object that contains the url (host + filename) to the video file.

To add video objects via web UI, navigate to /admin and create Video objects. Alternatively, use ./manage.py shell, and create annotator.Video objects and call video.save(). Helper methods exist to create large number of video objects at once, see annotator/models.py.

Video objects can either use H.264 encoded video (See scripts/convert-to-h264), or a list of frames provided in the attribute image_list in JSON format (e.g. video.image_list = '["20170728_085435.jpg"]'). By using single-frame videos, BeaverDam can be used for image annotation.

Video annotations can be accessed via admin, /annotation/video_id, or through the Video objects' annotation attribute through the shell.

Tasks

Tasks are created in the same way as Videos. Only the video attribute needs to be filled out at creation time. They can be published to mturk by calling task.publish().

Simulating mturk view in debug

To see what video pages look like on mturk preview mode, set url param preview=true. For mturk's HIT accepted mode, set url param mturk=true.

Example: localhost:5000/video/0/?mturk=true

MacOS uWSGI

For MacOS, you may need to do uWSGI==2.0.17

Running tests

Inside venv, run ./manage.py test

Contributing

Pull requests and contributions are welcome. See annotator/static/README.md for more info on frontend architecture.

Support

For help setting up BeaverDam for your application/company, please contact me or leave an issue.

beaverdam's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

beaverdam's Issues

Make "Thing editor"

We don't need to have a sidebar – that would be too much mouse movement.

Instead we can have settings in a popup:

  /---------------------------------------\
  | Edit thing #2                         |
  |---------------------------------------|
  |                                       |
  |  Type [ Car |*Pedestrian*| Other ]    |
  |                                       |
  |  [√] Hide when unset                  |
  |                                       |
  |  Notes ---------------------------+   |
  |  |                                |   |
  |  |                                |   |
  |  +--------------------------------+   |
  |                                       |
  |                             [Delete]  |
  \________________   ____________________/
                   \ /
                    V

This library can help with popups: http://github.hubspot.com/drop/docs/welcome/

Export frames with annotations

Great tool.

I need to export frames + annotations for each so I can train SSD (single shot detector).

It looks like you're storing keyframes + interpolating between them in the Web UI.. is there some existing way to export the interpolated annotations frame by frame?

Figure I'll have to write some code - just looking for help on a starting point. Thanks!

Mouseover icons for resizing

Currently, you can drag edges of selected boxes to resize them. We'd like to make it so that when the mouse is hovering over a border, the appropriate double arrow mouse icon is used (like in MS Word, for example). In CSS, this is done by changing the "cursor" property to "ns-resize", "ew-resize", "nesw-resize", and "nwse-resize". This new cursor should be used until the user is no longer mousing over a border of a selected box.

Allow smaller boxes

This means that we need to have a way to manipulate smaller boxes. Here's an idea:

2016-07-27 23 18 49

Running BeaverDam outside of Mturk

Hello. First of all thank you for making the installation process so simple. I was able to get this up and running in no time.

I was wondering though if it's possible to run BeaverDam outside of Mturk. Correct me if I'm wrong, but I should be able to host this somewhere access work from the browser, right? I couldn't figure out how I can upload new content and access output. Any help will be much appreciated.

Thanks,
Anuj

Tighten X-Frame-Options on mturk views

Create a decorator that replaces django.views.decorators.clickjacking.xframe_options_exempt that only exempts after verifying that the source is mturk (via assignmentId)

Finalize video DB

This includes:

  • Updating DB schema for Video
  • Have pipeline for loading new videos
  • Include utility scripts to place video on s3

Add way to ask users questions

We want to ask people questions such as:

  • Are there way too many cars to label?
  • Is there too much crap in the video to be used as training data?

Standalone

How can I use the beaverdam as a standalone tool to annotate my own videos and get the coordinates of my boxes?

Connecting to MechanicalTurk

Hi,

How do we connect this to mechanical turk? Is there some missing documentation?

Here's what we've done so far:

  1. Created an AWS account
  2. Created an AWS user with full turk access only
  3. Created environment variables with the AWS_ID and AWS_KEY set
  4. Created a fullvideotask in the admin site: /admin/mturk/fullvideotask/

I can see that the Task class in mturk/models.py actually calls publish and creates the HIT with AWS, but I can't see where the publish method is invoked.

Can you please provide some more instuctions?

Thanks for your help.

List of Things

In the event that two things overlap with each other, it may useful to have a list of things such that if you click the thing, it'll be selected and moved to the top of the box list.

Add move/drag shortcuts

Holding down one of these keys
Q W E
A S D
Z X C
while moving the mouse should have the same effect as click+dragging from that edge.

Suggested by @allenywang

Make UI responsive

Why:

  • All the controls should always be visible
  • Controls should be smartly placed to reduce mouse move distance
  • Lots of awkward blank space sometimes

Remove magic numbers

Lots of numbers are currently in the code from the width of borders and etc. Would be nice to get some help to remove them

adding video

It is pretty unclear how to add a video .
Some explicit instructions for those like me who are baffled after install:

  1. start the venv:
    source venv/bin/activate
  2. add your video to BeaverDam/annotator/static/videos (I think this is right, not that there is any documentation on it...)

After doing those I hit :

Exception Value:
unsupported operand type(s) for +: 'Exception' and 'str'
Exception Location: /home/jeremy/sw/BeaverDam/annotator/models.py in url, line 43

where line 43 is
raise Exception('Video {0} does not have a filename, host or
image_list. Possible fixes: \n1) Place {0}.mp4 into static/videos to serve locally. \n2) Update the filename & host fields of the Video with id={0}'.format(self.id)) + self.filename

So it appears there is a parenthesis problem here, the line should read

raise Exception('Video {0} does not have a filename, host or
image_list. Possible fixes: \n1) Place {0}.mp4 into static/videos to serve locally. \n2) Update the filename & host fields of the Video with id={0}'.format(self.id) + self.filename )

after fixing that in models.py, I get an error complaining about the filename. I will try changing to 149.mp4 as the error seems to expect that name.

Create duplicate box button to make non moving objects easier to track

Due to the interpolation between when the car is stopped at a light and when the car starts to move when the light turns green, the box drifts. Should have some sort of functionality to either duplicate the box exactly where it was so that there is no delta in between the two times.

Add tutorial

This includes a popup (or alternative) that has page by page instructions, and a button to re-open this popup.

Hotkey to move between frames while in focus

Need keypress functionality to move forward or backward to the next keyframe when focused. A keyframe expander would be nice too but not necessary if keypress functionality is implemented.

Skipping forward in video

Using the sample videos, the video player has the feature of allowing the annotator to skip forward to any time in the video (either using the bar or by typing in a time). This feature works for the videos hosted on S3, but for locally-stored videos (such as /video/0) it doesn't work when the video is first loaded - only after the page is refreshed.

Any suggestions as to how I can either fix this or work around it?

Add jshint setup to README

Draft:

Instructions for Sublime Text 3:

  1. Install Package Control
  2. Tools » Command Palette » Install Package » SublimeLinter
  3. Tools » Command Palette » Install Package » SublimeLinter-jshint
  4. npm install jshint

source:

Instructions for Sublime Text 3:

1. Install [Package Control](https://packagecontrol.io/installation)
2. Tools » Command Palette » Install Package » `SublimeLinter`
3. Tools » Command Palette » Install Package » `SublimeLinter-jshint`
4. `npm install jshint`

Gold standard worker verification

When the server passes a flag indicating this mode, the annotations loaded will not be displayed. Rather, the worker must annotate from scratch, and his work is compared to the loaded annotation at the end, cannot submit until they sufficiently match, and is encouraged to re-try.

Main challenge of this is writing an algorithm for "sufficiently match"

Enforce minimum box size

Create a global config parameter min_size, such that if the user tries to create any box's width or height is smaller that min_size, ignore it and do nothing.

Add a new video

I want to use the sample video inside the static/videos/o.mp4. How I should set set the host/filename/source.

Count inside edge of border as true box, not the center of the border

Currently, since the border size is a few pixels, the boxes stored in the annotation are 0.5 * border_width larger than they actually are, assuming annotators are using the inside of the border to inscribe objects (right?).

We can fix this by either accounting for the offset in the box itself, or just convert all the boxes when we save the annotation.

Add timers to report worker speed

We want to record the following times:

  • Time between script load to beginning of first box drawing
  • Time between first box drawing to submit

These should be sent back along with the annotation.

The backend should then parse this information, and store it if there is an associated Task object.

Need attribute support

It's not clear how to do attributes like Vatic supports. For example, if I annotate a traffic light, I might like to mark its color. If I use different labels, like trafficlight-red, trafficlight-green, then they will not be the same object ID, so I would have to merge the tracks. It is better to have a color attribute in addition to the label that can be red or green.

Redo random color chooser

The getRandomColor method in thing.js should be looked into, as it currently doesn't span a large amount of hue space. And as color is currently being used as the ID of the Thing, it should enforce uniqueness as well.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.