alenzenx / tracknetv3 Goto Github PK

View Code? Open in Web Editor NEW

51.0 3.0 8.0 80.55 MB

Beyond TrackNetV2 ,and "First" TrackNet using Attention

Python 100.00%

tracknet tracknetv2 tracknetv3 badminton badminton-track attention tracknet-attention

tracknetv3's Introduction

TrackNetV3 : beyond TrackNetV2 ,and First TrackNet using Attention

TrackNet is a semantic segmentation tracking network.

It has the most advanced accuracy in the current TrackNet system, and also has the best accuracy for Few Shot Learning!

擁有目前TrackNet追蹤網路中最先進的精準度，且對於小樣本同樣有最佳的精準度!

V2 and V3 are trained on the same small sample dataset (with the following distributions: training set = 32,421 images, validation set = 3,717 images, test set = 11,118 images).

V2 跟 V3 是在同樣的小樣本資料集訓練(以下 train set=32421張 valid set=3717張 test set=11118張)

Trainset : https://github.com/alenzenx/TrackNetV3/tree/main/raw_data

Testset : https://github.com/alenzenx/TrackNetV3/tree/main/raw_data2

Paper : Click to obtain TrackNetV3 paper

TrackNetV3 architecture

Key component

TrackNetV3 best accuracy in small sample dataset: 90.53%

Click to obtain training weights | 點擊以獲取權重 : 90.53%

model from : model.py

TrackNetV2 best accuracy in small sample dataset: 88.49%

model from : model原始檔.py

環境建議:

os system support :

```
windows 10 
```
```
windows 11
```
```
ubuntu
```
```
others Linux
```

python = 3.7.9~3.9.4

cuda = 11.7 or others

cudnn = 8.9.0 or others (在windows下cudnn裡的檔案請全部拖進cuda中，不然tensorflow可能會報錯)

安裝步驟 :

請先檢查有沒有 tutorial-env 的資料夾，有的話請先整個刪除，重新安裝虛擬環境

only Windows :

python 3.9.4安裝時要 Add Python 3.9 to PATH

tensorflow安裝前，請先在 Windows 上啟用長路徑

tensorflow安裝前，要下載並安裝Microsoft Visual C++ Redistributable for Visual Studio 2015、2017 和 2019 。

安裝虛擬環境 :

python -m venv tutorial-env

安裝pytorch(cuda11.7) :

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117

安裝tensorflow-GPU(大概會安裝tensorflow==2.10.0) :

pip install "tensorflow<2.11"

安裝其他套件 :

pip install -r requirements.txt

如何自己做數據集:

標註video:(會產生csv)

python imgLabel.py --label_video_path=你要標註的影片

使用此連結的標註方法

全部的影片都標註完成後，請自行分開訓練集與測試集(驗證集)

1.成對的訓練集:影片與csv檔丟到raw_data資料夾

2.成對的測試集(驗證集):影片與csv檔丟到raw_data2資料夾

注意! 如果剛下載專案raw_data資料夾與raw_data2已經存在檔案的話，代表我已經標註好了，你可以選擇使用我標註的直接訓練，也可以自行標註。

TrackNetV2_Dataset資料夾請保持下列形式:

    TrackNetV2_Dataset
                ├─ train
                |    
                |
                └─ test

注意! 除了上述的train與test，TrackNetV2_Dataset底下的其他檔案請在標註前都刪除(包括train底下的資料夾與test底下的資料夾)

轉換資料集的格式 : 運行 `zz_Tracknet_badminton_DataConvert.py`

如果是要轉換訓練集

請在 zz_Tracknet_badminton_DataConvert.py 裡更改

original_raw_data = 'raw_data'

target_folder = 'TrackNetV2_Dataset/train'

然後運行 python zz_Tracknet_badminton_DataConvert.py

如果是要轉換測試集(驗證集)

請在 zz_Tracknet_badminton_DataConvert.py 裡更改

original_raw_data = 'raw_data2'

target_folder = 'TrackNetV2_Dataset/test'

然後運行 python zz_Tracknet_badminton_DataConvert.py

轉換後檢查及預處理影像:

python preprocess.py

注意! 如果 TrackNetV2_Dataset 裡已經有訓練集與測試集(驗證集)且2個資料夾裡都有match1、match2...資料夾，即可開始訓練。

備註:

zz_Tracknet_badminton_DataConvert.py的檔案是將imgLabel.py生成的raw_data 轉換成 Tracknetv2-main 所需要的格式。因為原始資料的標註軟體沒給，所以用imgLabel.py代替，所以需要一個 zz_Tracknet_badminton_DataConvert.py 來轉格式。
在Windows下請不要用python3，請使用python

如何開始訓練:

請注意訓練時的batchsize，會關係到專屬GPU記憶體的大小，我使用的 GPU 是 Nvidia RTX3060 12G ，我使用的batchsize=4

訓練:

python train.py --num_frame 3 --epochs 30 --batch_size 4 --learning_rate 0.001 --save_dir exp

預測 :

python predict.py --video_file=test.mp4 --model_file=exp/model_best.pt --save_dir pred_result

預測後使用 : 去躁及 smooth 羽球預測的曲線

python denoise.py --input_csv=pred_result/test_ball.csv

smooth 羽球預測的曲線後 : predict 優化後的影片

python show_trajectory.py --video_file test.mp4 --csv_file pred_result/test_ball.csv --save_dir pred_result

smooth 羽球預測的曲線後 : 預測擊球時刻

python event_detection.py --input_csv=pred_result/test_ball.csv

reference

TrackNetV3 是參考原始版本的 TrackNetV2 及 Resnet 版本的 TrackNet-Badminton-Tracking-tensorflow2 加以改進的TrackNet版本。

tracknetv3's People

Contributors

Stargazers

Watchers

Forkers

marcol3786 alexdpoon minlianglin jeffliu123 qiuboxiang rebots-online fengsxy zzambia

tracknetv3's Issues

Can't I skip the training step?

Dear author,
I hope this email finds you well. I am currently working with your video processing scripts for my project. First and foremost, I want to express my gratitude for providing such a valuable resource.

However, I have encountered a couple of issues while running the scripts, and I would greatly appreciate your guidance in resolving them.

Error in denoise.py:
When executing the denoise.py script, I encountered the following error:

Details

Traceback (most recent call last): File "C:\Users\Hank\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\core\indexes\base.py", line 3361, in get_loc return self._engine.get_loc(casted_key) File "pandas\_libs\index.pyx", line 76, in pandas._libs.index.IndexEngine.get_loc File "pandas\_libs\index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc File "pandas\_libs\hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas\_libs\hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'X'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "denoise.py", line 557, in
smooth(os.path.join(rootdir, 'pred_result', i))
File "denoise.py", line 9, in smooth
x = df['X'].tolist()
File "C:\Users\Hank\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\core\frame.py", line 3458, in getitem
indexer = self.columns.get_loc(key)
File "C:\Users\Hank\AppData\Local\Programs\Python\Python37\lib\site-packages\pandas\core\indexes\base.py", line 3363, in get_loc
raise KeyError(key) from err
KeyError: 'X'

2.Error in predict.py:
When running the predict.py script with a test video consisting of 935 frames (possibly a multiple of 5), I encountered the following error:

935
Number of sampled frames: 0
Number of sampled frames: 15
Number of sampled frames: 30
Number of sampled frames: 45
Number of sampled frames: 60
Number of sampled frames: 75
Number of sampled frames: 90
Number of sampled frames: 105
Number of sampled frames: 120
Number of sampled frames: 135
Number of sampled frames: 150
Number of sampled frames: 165
Number of sampled frames: 180
Number of sampled frames: 195
Number of sampled frames: 210
Number of sampled frames: 225
Number of sampled frames: 240
Number of sampled frames: 255
Number of sampled frames: 270
Number of sampled frames: 285
Number of sampled frames: 300
Number of sampled frames: 315
Number of sampled frames: 330
Number of sampled frames: 345
Number of sampled frames: 360
Number of sampled frames: 375
Number of sampled frames: 390
Number of sampled frames: 405
Number of sampled frames: 420
Number of sampled frames: 435
Number of sampled frames: 450
Number of sampled frames: 465
Number of sampled frames: 480
Number of sampled frames: 495
Number of sampled frames: 510
Number of sampled frames: 525
Number of sampled frames: 540
Number of sampled frames: 555
Number of sampled frames: 570
Number of sampled frames: 585
Number of sampled frames: 600
Number of sampled frames: 615
Number of sampled frames: 630
Number of sampled frames: 645
Number of sampled frames: 660
Number of sampled frames: 675
Number of sampled frames: 690
Number of sampled frames: 705
Number of sampled frames: 720
Number of sampled frames: 735
Number of sampled frames: 750
Number of sampled frames: 765
Number of sampled frames: 780
Number of sampled frames: 795
Number of sampled frames: 810
Number of sampled frames: 825
Number of sampled frames: 840
Number of sampled frames: 855
Number of sampled frames: 870
Number of sampled frames: 885
Number of sampled frames: 900
Number of sampled frames: 915
Number of sampled frames: 930
Traceback (most recent call last):
  File "predict.py", line 105, in <module>
    assert len(frame_queue) % num_frame == 0
AssertionError

Is there any assistance or suggestions you could provide would be immensely helpful. Thank you for your time and consideration.

modify frame num is possible?

Using a 3-frame input for one deep learning CV model makes it difficult to integrate with other models. Can I change the num_frame parameter from 3 to 1 during training and then use the model ?

thk u

can shuttlecock be tracked in realtime in live match using tracknetv3 if that is possible how?

Is there a tennis model?

Hi! I wanna use this 3rd version of tracknet for detecting tennis balls. Are there any trained weights for tennis like it used to be in TrackNetV1?

How to use A100 GPU to train the data?

Hello @alenzenx ,

我在使用colab上80G顯卡訓練時，使用python train.py --num_frame 3 --epochs 30 --batch_size 24 --learning_rate 0.001 --save_dir exp報錯，大致意思是get_num=0，請問你知道怎麼發揮大顯卡的優勢，讓訓練的時間減短嗎？

When I was using the 80G graphics card for training on Colab, running python train.py --num_frame 3 --epochs 30 --batch_size 24 --learning_rate 0.001 --save_dir exp caused an error, roughly meaning get_num=0. Do you know how to take advantage of a powerful graphics card to reduce training time?

Colab Notebook

Hi,

I wanted to try this model out and hence I created a colab notebook.

The notebook is functional as of now. You can use it to visualize how the model performs on your matches.

To me personally It was a little disappointing as it leaves the ball on the far side of the court. Currently I am using TracknetV1 with 3 frame mode and It does have a lot of false positives but it doesn't miss ball coordinates as much.

Colab Notebook

Here's the video I tried. If @alenzenx has any ideas on how I can improve the ball detection on the far side of the court I am all ears!

4.Traj.mp4

hello，can you provide the model file ?

missing License ?

Hi,

Thanks for the amazing work you have done here. 👍

I wanted to ask about the usage, I can't see any LICENSE file.
So, how is it about using your code or your pre-trained models ?

I see that the popular ResNet version is under MIT.

About dataset

Dear author,
I want to ask u the if dataset in the repo is different with the dataset in the tracknetv2?
Looking forward to your reply!