sforaidl / playground Goto Github PK

Run the code, after making the changes mentioned in the README for the Windows operating system, and verify that everything is working as intended. If there is some error anywhere submit a new issue.

Optimization code has been done on detect_players_video. Add the same for heatmap too. Preferably create helper functions for the same in utils and call them in both detect players video and get heatmap instead of having redundant code in both

Differentiate between teams based on color of jersey. We can even discuss other approaches. What do you think? @Rijul1999

A good starting point can be the work from https://github.com/simonefrancia/SpaceJam

Both for heatmap and videos, multiple progress bars are displayed. There should be just one progress bar.

For the get_heatmap and detect_players_video methods, create a progress bar/ show ETA dynamically so that the user can know how much time before the entire video will be processed.

We can start with this paper https://www.cs.toronto.edu/~davidj/projects/towards_real_time_detection_tracking.pdf. We can also coordinate with #91 in order to build some sort of common pipeline for tracking players in sports.

Creating and updating documentation for the methods in the Detector module.

Save figure flag in the detector has not been implemented yet. With an argument similar to display_detection, allow an argument for save figure or save prediction.

Based on the direction of motion the bat of a batsman, draw lines on the wagon wheel to analyze where a batsman tends to hit. This can help in better field positions.

Create helper functions in utils.py for modularising the code in badminton.py. Currently, the code seems a little too cluttered and congested.

When the players are detected in the first frame, instead of running detections on all frames thereafter, we need to track the players. See the link for example https://drive.google.com/file/d/1FwGkApZwX03mCpvA1BD1aRT1nVa6QMnZ/view?usp=sharing

The dataset has been uploaded. We'll need to extract frames and annotations and prepare the directory format for our models. The eaf files contain the starting and ending points of actions along with their annotations. Since there are multiple eaf files, I'd suggest taking up one video/eaf file and processing that first to understand if your code is working fine. Please download the video files themselves from YouTube, since they are too large by themselves to be added to the repository. The expected Directory structure after processing is:

-Data
-------train
-------------smash
---------------------clip1
-----------------------------frame1
-----------------------------frame2
-----------------------------frame3
-----------------------------and so on
---------------------clip2
-----------------------------frame1
-----------------------------frame2
-----------------------------frame3
-----------------------------and so on
-------------lob and so on

-------val
------------Similarly

Badminton.py has a lot of redundant code in terms of checks and if-else statements.
For example:

if optimization==True:
            frames_skipped=frames_skipped_input
        else:
            frames_skipped=1
if optimization==True:
     count_of_frames=0
      .. and so on

Such scenarios can be removed and code can be optimised further

Adding and implementing the work from:
https://www.researchgate.net/publication/324759769_Pose_Trainer_Correcting_Exercise_Posture_using_Pose_Estimation

Add docs for heatmap generations and usage

The bottom center of the detected person must be extracted and returned which will be used for heatmap generation. Simply return those co-ordinates from the detect_players method.

Currently, the heatmap base image contains the players too. We would like to have a base template court of the same size as the wrapped transform(see dimensions of heatmap.png) and show heatmap on that rather than a court image with players.
Eg: If we can have a pure white image with just badminton court "white" lines in black as base image, that would be good to show the heatmap on. We can explore other colors and black/neon color scenes later

Using get heatmap, extra progress bars can be seen both in the optimization case and no optimization case.

FPS issues were partially solved with the usage of tiny yolo but it comes at the cost of Detector performance / maP. Check/implement on tracking strategies to improve the detector for tiny yolo.

Implementing point segmentation. A good starting point would be the approach from the paper:
http://cdn.iiit.ac.in/cdn/cvit.iiit.ac.in/images/ConferencePapers/2018/badminton_analytics.pdf

However, open to better approaches if known.

Currently weights need to be explicitly downloaded beforehand using the script config/download_weights.sh
Add necessary code in Detector in Badminton/Badminton.py to check if weights file has been downloaded, if no, download it within code and give error is weights cannot be downloaded(can happen due to network, space or some other issue)

We can make use of existing implementations of I3D/CSNs from https://github.com/open-mmlab/mmaction

Add docs for recent optimization code

A good starting point would be to use the intensity (pitch / something else) of audio from commentary to filter out "exciting events" and compile the clips from those events to generate highlight.

With the latest merge, we are no more rendering the display for heatmap and detect players video, we can have an option to render. However, I'm not sure if it's needed. I would like to hear some of your comments on it!

Using homography or some other technique, conversion of the plane of view as seen in the sample output image to a top view image of court for heatmap generation.

Currently, the detect_players method takes in image path as parameter. Modify it to load the image beforehand and take the loaded image as parameter.

Perhaps different users have different indentation specifications which is causing these inconsistent tab errors.

  File "<ipython-input-20-25d4a5813df5>", line 183
    ret,frame = cap.read()
                          ^
TabError: inconsistent use of tabs and spaces in indentation

The tiny yolo module has been implemented but the cfg file is not downloaded

obj = Detector(tiny=True)

/content/Playground/Badminton/utilities/parse_config.py in parse_model_config(path)
      1 def parse_model_config(path):
      2     """Parses the yolo-v3 layer configuration file and returns module definitions"""
----> 3     file = open(path, "r")
      4     lines = file.read().split("\n")
      5     lines = [x for x in lines if x and not x.startswith("#")]

FileNotFoundError: [Errno 2] No such file or directory: 'Badminton/config/yolov3-tiny.cfg'

Currently, there's some redundant/unnecessary code because of which FPS of video detector is very poor. Making optimizations to improve FPS is necessary.

dataset extractor is complete. Work on action segmentation can be started

Currently, image formatting is slightly off with different parts of code using images loaded by cv2 and some using images loaded by PIL. Since the formats are different, there are some bugs. We will stick to OpenCV for image loading and all image operations from now on so changing PIL stuff wherever possible to OpenCV.

Using the standard pytorch detector, detect the players and football from football videos.

Set up a new env and generate requirements.txt after installing all the necessary dependencies like cv2, torch etc. and export requirements.txt

In the detector, check if the input is image or video. If it is a video then parse through the video and run the detector on each of the individual frames of the video.