tteepe / tracktacular Goto Github PK

View Code? Open in Web Editor NEW

12.0 1.0 1.0 788 KB

Official Code for "Lifting Multi-View Detection and Tracking to the Bird’s Eye View"

Home Page: https://arxiv.org/abs/2403.12573

Python 100.00%

multi-view tracking

tracktacular's Introduction

TrackTacular 🐙

Lifting Multi-View Detection and Tracking to the Bird’s Eye View

Torben Teepe, Philipp Wolters, Johannes Gilg, Fabian Herzog, Gerhard Rigoll

Tip

This work is an extension of your previous work EarlyBird 🦅. Feel free to check it out and extend our multi-view object detection and tracking pipeline on other datasets!

Usage

Getting Started

Install PyTorch with CUDA support

pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu118

Install mmcv with CUDA support

pip install mmcv==2.0.0 -f https://download.openmmlab.com/mmcv/dist/cu118/torch2.1/index.html

Install remaining dependencies
```
pip install -r requirements.txt
```

Training

python world_track.py fit -c configs/t_fit.yml \
    -c configs/d_{multiviewx,wildtrack,synthehicle}.yml \
    -c configs/m_{mvdet,segnet,liftnet,bevformer}.yml

Testing

python world_track.py test -c model_weights/config.yaml \
    --ckpt model_weights/model-epoch=35-val_loss=6.50.ckpt

Acknowledgement

Simple-BEV: Adam W. Harley
MVDeTr: Yunzhong Hou

Citation

@article{teepe2023lifting,
      title={Lifting Multi-View Detection and Tracking to the Bird's Eye View}, 
      author={Torben Teepe and Philipp Wolters and Johannes Gilg and Fabian Herzog and Gerhard Rigoll},
      year={2024},
      eprint={2403.12573},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

tracktacular's People

Contributors

Stargazers

Watchers

Forkers

minho8849

tracktacular's Issues

Model evaluation

Thank you for your work and contributions to the field.

I am evaluating the model but I am getting the following error:
dt_dets = dt[np.logical_and(dt[:, 0] == seq, dt[:, 1] == frame)][:, (2, 8, 9)]
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed
Testing DataLoader 0: 100%|██████████| 40/40 [00:12<00:00, 3.23it/s]

I appreciate your assistances
Best regards,

How to decide the parameter in yml file?

Than you for sharing your great work and implementation!
I have some quenstions.
How to decide zmax in bounds, Z in resolution and z_sign in data yml file?
(e.g., bounds: [0, 1000, 0, 640, 0, **2**] , z_sign: **-1** in d_multiviewx.yml)

Pre-train model Performance

Hi,
Thanks for this great work. I have tried to use pre-train model but get very low performance. Would you please help me to find out where I made mistake.

I ran with this command.
python world_track.py test -c model_weights/wild_segnet/config.yaml
--ckpt model_weights/wild_segnet/model-epoch=21-val_loss=7.79-val_center=4.76.ckpt

And got this values:
IDF1 IDP IDR Rcll Prcn GT MT PT ML FP FN IDs FM MOTA MOTP IDt IDa IDm
0 1.2% 31.6% 0.6% 0.6% 31.6% 41 0 0 41 13 946 0 0 -0.7% 0.566 0 0 0
OVERALL 1.2% 31.6% 0.6% 0.6% 31.6% 41 0 0 41 13 946 0 0 -0.7% 0.566 0 0 0
Testing DataLoader 0: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 40/40 [00:30<00:00, 1.30it/s]
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Test metric ┃ DataLoader 0 ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ detect/moda │ 0.0 │
│ detect/modp │ 34.748539923465074 │
│ detect/precision │ 4.545454545454546 │
│ detect/recall │ 0.20491803278688525 │
│ track/idf1 │ 1.2358393669128418 │
│ track/idp │ 31.578947067260742 │
│ track/idr │ 0.6302521228790283 │
│ track/mostly_lost │ 1.0 │
│ track/mostly_tracked │ 0.0 │
│ track/mota │ -0.7352941036224365 │
│ track/motp │ 43.35531234741211 │
│ track/num_ascend │ 0.0 │
│ track/num_false_positives │ 13.0 │
│ track/num_fragmentations │ 0.0 │
│ track/num_migrate │ 0.0 │
│ track/num_misses │ 946.0 │
│ track/num_switches │ 0.0 │
│ track/num_transfer │ 0.0 │
│ track/num_unique_objects │ 41.0 │
│ track/partially_tracked │ 0.0 │
│ track/precision │ 31.578947067260742 │
│ track/recall │ 0.6302521228790283 │
└───────────────────────────┴───────────────────────────┘

data split

Thanks for sharing your work,
i have a question regarding the split of the dataset:

in 'PedestrianDataModule' you have:

    if stage == 'fit':
      self.data_train = PedestrianDataset(
          base,
          is_train=True,
          resolution=self.resolution,
          bounds=self.bounds,
      )

  if stage == 'fit' or stage == 'validate':
      self.data_val = PedestrianDataset(
          base,
          is_train=False,
          resolution=self.resolution,
          bounds=self.bounds,
      )

  if stage == 'test':
      self.data_test = PedestrianDataset(
          base,
          is_train=False,
          resolution=self.resolution,
          bounds=self.bounds
      )

  if stage == 'predict':
      self.data_predict = PedestrianDataset(
          base,
          is_train=False,
          resolution=self.resolution,
          bounds=self.bounds,
      )`

dose this means that the test and validation data are the same?