Giter VIP home page Giter VIP logo

Comments (11)

lawkane avatar lawkane commented on May 27, 2024 1

@yelantf Thank you very much. I check and found the img_id in ava_val_det_person_boxes is not consistent with those in ava_val_v2.2_min.json. That's what you said. Now the val video could be loaded.
Thanks again!

from alphaction.

yelantf avatar yelantf commented on May 27, 2024

There is no "ava_train_det_person_bbox.json" in our pipeline because we train with the bounding boxes in the converted ground-truth annotation file "ava_train_v2.2_min.json". If you have done step 5 in DATA.md, this file should be generated.

However, according to your description, it seems that you are referring to "ava_val_det_person_bbox.json". Then the answer is, yes, you should replace those files listed in step6.
For training, you should replace the "ava_train_det_object_bbox.json" by running the object detector on your custom dataset. For validation, you should run the human detector to replace "ava_val_det_person_bbox.json" and the object detector to replace "ava_val_det_object_bbox.json". The object JSON files are necessary for our IA structure.

from alphaction.

lawkane avatar lawkane commented on May 27, 2024

@yelantf
Thanks for your reply and sorry for my erroneous description.
Yes, I have done step 5 in DATA.md. Now I try to replace the files listed in step 6, but I can't find the guidance about running the detector to generate the new 'bbox.json'. Could you tell me how to run the object detector and human detector on my custom dataset?
Thanks again.

from alphaction.

yelantf avatar yelantf commented on May 27, 2024

We use maskrcnn-benchmark for detection tasks. The generated "ava_train_v2.2_min.json", "ava_val_v2.2_min.json" both have COCO format and could be used as their annotation files. Please refer to https://github.com/facebookresearch/maskrcnn-benchmark for detailed instructions.

from alphaction.

lawkane avatar lawkane commented on May 27, 2024

@yelantf
Thank you very much for your answer. I will try to do this.

from alphaction.

lawkane avatar lawkane commented on May 27, 2024

@yelantf Hi, the author.Now I have a new problem.
I prepared the dataset and run the command python -m torch.distributed.launch --nproc_per_node=8 \ train_net.py --config-file "config_files/resnet50_4x16f_denseserial.yaml" \ --transfer --no-head --use-tfboard
Then I got the error:

2020-12-01 22:20:13,839 alphaction.inference INFO: Start evaluation on ava_video_val_v2.2 dataset(0 videos).
Traceback (most recent call last):
File "train_net.py", line 245, in
main()
File "train_net.py", line 235, in main
args.no_head)
File "train_net.py", line 98, in train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 134, in do_train
mem_active
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 166, in val_in_train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 170, in inference
predictions = compute_on_dataset(model, data_loader, device, logger, mem_active)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 127, in compute_on_dataset
results_dict = compute_on_dataset_2stage(model, data_loader, device, logger)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 53, in compute_on_dataset_2stage
loader_len = len(data_loader)
File "/root/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 397, in len
return len(self._index_sampler)
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 107, in len
self._batches = self._prepare_batches()
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 50, in _prepare_batches
order[sampled_ids] = torch.arange(len(sampled_ids))
IndexError: tensors used as indices must be long, byte or bool tensors
Traceback (most recent call last):
File "train_net.py", line 245, in
main()
File "train_net.py", line 235, in main
args.no_head)
File "train_net.py", line 98, in train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 134, in do_train
mem_active
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 166, in val_in_train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 170, in inference
predictions = compute_on_dataset(model, data_loader, device, logger, mem_active)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 127, in compute_on_dataset
results_dict = compute_on_dataset_2stage(model, data_loader, device, logger)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 53, in compute_on_dataset_2stage
loader_len = len(data_loader)
File "/root/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 397, in len
return len(self._index_sampler)
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 107, in len
self._batches = self._prepare_batches()
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 50, in _prepare_batches
order[sampled_ids] = torch.arange(len(sampled_ids))
IndexError: tensors used as indices must be long, byte or bool tensors
Traceback (most recent call last):
File "train_net.py", line 245, in
main()
File "train_net.py", line 235, in main
args.no_head)
File "train_net.py", line 98, in train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 134, in do_train
mem_active
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 166, in val_in_train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 170, in inference
predictions = compute_on_dataset(model, data_loader, device, logger, mem_active)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 127, in compute_on_dataset
results_dict = compute_on_dataset_2stage(model, data_loader, device, logger)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 53, in compute_on_dataset_2stage
loader_len = len(data_loader)
File "/root/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 397, in len
return len(self._index_sampler)
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 107, in len
self._batches = self._prepare_batches()
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 50, in _prepare_batches
order[sampled_ids] = torch.arange(len(sampled_ids))
IndexError: tensors used as indices must be long, byte or bool tensors
Traceback (most recent call last):
File "train_net.py", line 245, in
main()
File "train_net.py", line 235, in main
args.no_head)
File "train_net.py", line 98, in train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 134, in do_train
mem_active
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 166, in val_in_train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 170, in inference
predictions = compute_on_dataset(model, data_loader, device, logger, mem_active)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 127, in compute_on_dataset
results_dict = compute_on_dataset_2stage(model, data_loader, device, logger)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 53, in compute_on_dataset_2stage
loader_len = len(data_loader)
File "/root/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 397, in len
return len(self._index_sampler)
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 107, in len
self._batches = self._prepare_batches()
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 50, in _prepare_batches
order[sampled_ids] = torch.arange(len(sampled_ids))
IndexError: tensors used as indices must be long, byte or bool tensors
Traceback (most recent call last):
File "train_net.py", line 245, in
main()
File "train_net.py", line 235, in main
args.no_head)
File "train_net.py", line 98, in train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 134, in do_train
mem_active
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 166, in val_in_train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 170, in inference
predictions = compute_on_dataset(model, data_loader, device, logger, mem_active)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 127, in compute_on_dataset
results_dict = compute_on_dataset_2stage(model, data_loader, device, logger)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 53, in compute_on_dataset_2stage
loader_len = len(data_loader)
File "/root/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 397, in len
return len(self._index_sampler)
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 107, in len
self._batches = self._prepare_batches()
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 50, in _prepare_batches
order[sampled_ids] = torch.arange(len(sampled_ids))
IndexError: tensors used as indices must be long, byte or bool tensors
Traceback (most recent call last):
File "train_net.py", line 245, in
main()
File "train_net.py", line 235, in main
args.no_head)
File "train_net.py", line 98, in train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 134, in do_train
mem_active
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 166, in val_in_train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 170, in inference
predictions = compute_on_dataset(model, data_loader, device, logger, mem_active)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 127, in compute_on_dataset
results_dict = compute_on_dataset_2stage(model, data_loader, device, logger)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 53, in compute_on_dataset_2stage
loader_len = len(data_loader)
File "/root/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 397, in len
return len(self._index_sampler)
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 107, in len
self._batches = self._prepare_batches()
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 50, in _prepare_batches
order[sampled_ids] = torch.arange(len(sampled_ids))
IndexError: tensors used as indices must be long, byte or bool tensors
Traceback (most recent call last):
File "train_net.py", line 245, in
main()
File "train_net.py", line 235, in main
args.no_head)
File "train_net.py", line 98, in train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 134, in do_train
mem_active
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 166, in val_in_train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 170, in inference
predictions = compute_on_dataset(model, data_loader, device, logger, mem_active)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 127, in compute_on_dataset
results_dict = compute_on_dataset_2stage(model, data_loader, device, logger)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 53, in compute_on_dataset_2stage
loader_len = len(data_loader)
File "/root/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 397, in len
return len(self._index_sampler)
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 107, in len
self._batches = self._prepare_batches()
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 50, in _prepare_batches
order[sampled_ids] = torch.arange(len(sampled_ids))
IndexError: tensors used as indices must be long, byte or bool tensors
Traceback (most recent call last):
File "train_net.py", line 245, in
main()
File "train_net.py", line 235, in main
args.no_head)
File "train_net.py", line 98, in train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 134, in do_train
mem_active
File "/root/luohan/AlphAction/alphaction/engine/trainer.py", line 166, in val_in_train
mem_active,
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 170, in inference
predictions = compute_on_dataset(model, data_loader, device, logger, mem_active)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 127, in compute_on_dataset
results_dict = compute_on_dataset_2stage(model, data_loader, device, logger)
File "/root/luohan/AlphAction/alphaction/engine/inference.py", line 53, in compute_on_dataset_2stage
loader_len = len(data_loader)
File "/root/.local/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 397, in len
return len(self._index_sampler)
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 107, in len
self._batches = self._prepare_batches()
File "/root/luohan/AlphAction/alphaction/dataset/samplers/grouped_batch_sampler.py", line 50, in _prepare_batches
order[sampled_ids] = torch.arange(len(sampled_ids))
IndexError: tensors used as indices must be long, byte or bool tensors
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/root/.local/lib/python3.7/site-packages/torch/distributed/launch.py", line 260, in
main()
File "/root/.local/lib/python3.7/site-packages/torch/distributed/launch.py", line 256, in main
cmd=cmd)
subprocess.CalledProcessError: Command '['/opt/conda/bin/python', '-u', 'train_net.py', '--local_rank=7', '--config-file', 'config_files/test.yaml', '--transfer', '--no-head', '--use-tfboard']' returned non-zero exit status 1.

I have made the "ava_val_v2.2.csv" and generated "ava_train_v2.2_min.json", "ava_val_v2.2_min.json".But it was unsuccessfully to start evaluation?
Thanks before for your reply.

from alphaction.

yelantf avatar yelantf commented on May 27, 2024

It shows that your dataset is empty, please ensure ava_val_v2.2_min.json is not empty and ava_val_det_person_bbox.json is correctly prepared.

2020-12-01 22:20:13,839 alphaction.inference INFO: Start evaluation on ava_video_val_v2.2 dataset(0 videos).

from alphaction.

lawkane avatar lawkane commented on May 27, 2024

@yelantf I am sure that my "ava_val_v2.2_min.json" is not empty. I run the object detector and person detector on my dataset and got the "ava_val_det_person_bbox.json".The bounding box json is correctly prepared, but I still got the error.
Besides, I tried to change the configure file that
DATASETS: TRAIN: ("ava_video_val_v2.2",)
It was successful to run the train which show that my 'ava_val_v2.2_min.json' is not empty.
Now I am quite confused.

from alphaction.

yelantf avatar yelantf commented on May 27, 2024

So you can train with it but cannot do inference? The difference between training and inference is that during training we use the boxes in the annotation file, while during inference we use pre-computed bounding boxes. I think there may be some problem in your ava_val_det_person_boxes (for example, the img_id in it is not consistent with those in ava_val_v2.2_min.json). I think it should not be so hard to locate the problem, you can add some print in the dataset class to see why it becomes empty.

from alphaction.

lawkane avatar lawkane commented on May 27, 2024

@yelantf Thanks for you advice. I add the print and found that during the function make_data_loader, the return 'data_loaders_val' is empty.Is 'ava_val_v2.2_min.json' relevant to the 'ava_video_val_v2.2.csv' when executed the make_data_loader?

from alphaction.

yelantf avatar yelantf commented on May 27, 2024

Yes, it is. I mean, you could add print in the dataset class defined in following lines. json_dict is the loaded json, clip_ids is the id numbers of every clip that will be used in the dataset. You can check when clip_ids gets empty with print.

AlphAction/alphaction/dataset/datasets/ava.py

Lines 64 to 140 in db19364

class AVAVideoDataset(data.Dataset):
def __init__(self, video_root, ann_file, remove_clips_without_annotations, frame_span, box_file=None, eval_file_paths={},
box_thresh=0.0, action_thresh=0.0, transforms=None, object_file=None, object_transforms=None,):
print('loading annotations into memory...')
tic = time.time()
json_dict = json.load(open(ann_file, 'r'))
assert type(json_dict) == dict, 'annotation file format {} not supported'.format(type(json_dict))
print('Done (t={:0.2f}s)'.format(time.time() - tic))
self.video_root = video_root
self.transforms = transforms
self.frame_span = frame_span
# These two attributes are used during ava evaluation...
# Maybe there is a better implementation
self.eval_file_paths = eval_file_paths
self.action_thresh = action_thresh
clip2ann = defaultdict(list)
if "annotations" in json_dict:
for ann in json_dict["annotations"]:
action_ids = ann["action_ids"]
one_hot = np.zeros(81, dtype=np.bool)
one_hot[action_ids] = True
packed_act = np.packbits(one_hot[1:])
clip2ann[ann["image_id"]].append(dict(bbox=ann["bbox"], packed_act=packed_act))
movies_size = {}
clips_info = {}
for img in json_dict["images"]:
mov = img["movie"]
if mov not in movies_size:
movies_size[mov] = [img["width"], img["height"]]
clips_info[img["id"]] = [mov, img["timestamp"]]
self.movie_info = NpInfoDict(movies_size, value_type=np.int32)
clip_ids = sorted(list(clips_info.keys()))
if remove_clips_without_annotations:
clip_ids = [clip_id for clip_id in clip_ids if clip_id in clip2ann]
if box_file:
# this is only for validation or testing
# we use detected boxes, so remove clips without boxes detected.
imgToBoxes = self.load_box_file(box_file, box_thresh)
clip_ids = [
img_id
for img_id in clip_ids
if len(imgToBoxes[img_id]) > 0
]
self.det_persons = NpBoxDict(imgToBoxes, clip_ids,
value_types=[("bbox", np.float32), ("score", np.float32)])
else:
self.det_persons = None
if object_file:
imgToObjects = self.load_box_file(object_file)
self.det_objects = NpBoxDict(imgToObjects, clip_ids,
value_types=[("bbox", np.float32), ("score", np.float32)])
else:
self.det_objects = None
if object_transforms:
self.object_transforms = object_transforms
else:
self.object_transforms = None
self.anns = NpBoxDict(clip2ann, clip_ids, value_types=[("bbox", np.float32), ("packed_act", np.uint8)])
clips_info = {
clip_id:
[
self.movie_info.convert_key(clips_info[clip_id][0]),
clips_info[clip_id][1]
] for clip_id in clip_ids
}
self.clips_info = NpInfoDict(clips_info, value_type=np.int32)

from alphaction.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.