wmcnally / evopose2d Goto Github PK

Again, amazing work.

I'm testing out the training and it appears that during the training itself, the preprocessing function is not being used (print statements not showing, train step mapping to three inputs, and not five). Is the prepressing happening on the tf backend and the preprocessing function primarily used for the demo? Any help here would be greatly appreciated.

Thanks again and sorry for all the issues.

this evopose model is very useful~!thanks

I'm confused by these lines in the dataloader.preprocess function:

    aspect_ratio = DATASET.INPUT_SHAPE[1] / DATASET.INPUT_SHAPE[0]
    if w > aspect_ratio * h:
        h = w / aspect_ratio
    scale = (h * 1.25) / DATASET.INPUT_SHAPE[0]

It seems to me that DATASET.INPUT_SHAPE[1] and [0] are swapped. This code would only make sense to if the dimensions in the coco.py config file are given as (height, width) which would be a strange departure from convention. Is there a mistake here, or am I misinterpreting something?

May I know the inference time of the model?

I have error AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'values'.

`$ python3 validate.py -c evopose2d_M_f32.yaml'

'Running on CPU
Number of accelerators: 1
Traceback (most recent call last):
File "validate.py", line 142, in
AP = validate(strategy, cfg, split='val')
File "validate.py", line 73, in validate
scores = np.concatenate(scores.values, axis=0)
AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'values'
`

How can i solve this problem?

Thank you for your nice work and I run the demo code,the input image include several person obviously,but only one person's pose keypoint could estimated.So i want to know if your model support multi-person pose estimation,thx!

during inference the model runs at 32-36 fps and not more than that on 2080ti

The cfg given in coco.py has

cn.DATASET.INPUT_SHAPE = [256, 192, 3]

This is the correct size for evopose2d_XS_f32.h5, but not for the three larger models (S, M, and L). Of course, one can edit coco.py to put the correct input sizes in, but I'm not sure if there are other differences in the configurations for the four models.

Could you provide the other training configs? Also, the demo currently says to download any of the four models, but it seems that XS is the one that should be recommended to avoid these size mismatches with the cfg.

Amazing contribution to the community.

Do you believe this method will perform similarly well for fine grained keypoints such as coco-whole body (which has hands and facial keypoints), or would that require another set of neuro evolution?

Thank you kindly for your input.

How do I train on a custom dataset? Is there any template?

Hi,
You've shared the json files of results. Would you please share the hrnet-w48 result with us too?
So thanks

I want to run the demo in win or linux.
How do I operate an image or video?

Hello,
Could you add a license file.

使用tf2onnx API:
tf2onnx.convert.from_keras()
将模型转为onnx格式后，发现模型输出的尺寸是输入的1/2，而不是1/4
用Netron查看onnx格式的模型，发现结构里有4次strides=2的卷积(3个depthwise + 1个常规)，3次ConvTranspose，但是配置文件里input_shape和output_shape却是按4倍降采样的设置......

I have tried to set batchsize 1 but still can see the memory usage slowly increasing until the process gets killed without any error or warning.

Sorry for flooding with issues. I can't find too much information regarding the data augmentation scheme. Specifically, I'm looking for joint symmetry and how this is handled in image mirroring, and adding more rotations (ie. 0-360 deg). Is this something that is parameterized in the repo? Thanks again for your help and amazing work.

In write_tfrecords, the dict passed to tf_example contains the path relative to args.coco_path, but tf_example is trying to access it from the working directory. I fixed this by passing coco_path to tf_example and changing line 30 to:

image_raw = open(coco_path + "/annotations/" + sample_dict['img_path'], 'rb').read()

which I believe should always work if coco_path is correct and the data is organized correctly.

I imagine there may be a better solution than changing all the tf_exaple function calls to accept coco_path but this was a quick and easy fix for my purposes.

I'm trying to train this model on Cloud TPUs and am wondering if you had to do anything special with the VM in order for this to work. I have it working fine on the local machine, but when I try to train on google cloud with TPU nothing happens. Any advice or documentation that you followed would be greatly appreciated.

Again, amazing work.

Do you believe that sigma (when generating heatmaps) will have any effect on output results? To me, the value creates heatmaps that are quite large and possibly overlap the mirrored joint (such as in the eyes).

Have you run any ablations on this value?

Is there any chance to contribute to this project to make it even better for the community? @wmcnally

To run demo.py, the the coco data requires an annotations folder including person_keypoints_val2017.json. This can be obtained from the 2017 Train/Val annotations [241MB] zip file at https://cocodataset.org/#download, but is not included in the images or person detections json from steps 5 and 6 in the readme.

Again, thank you for all the hard work. Once the model is trained on the TPU with float16, what is the process to convert these to float 32? Or does the entire model need to be trained as float32 in order to run as float 32?

I created a custom keypoint dataset with MS COCO format and then created tfrecords and then change yaml files, but when training I faced the following error:

Running on single GPU /device:GPU:0
Number of accelerators: 1
Training hrnet_w32_256x192 (28.59M / 7.65G) on GPU/CPU for 200 epochs
Traceback (most recent call last):
File "train.py", line 188, in
model, meta_data = train(strategy, cfg)
File "train.py", line 121, in train
train_step(train_iterator)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 780, in call
result = self._call(*args, **kwds)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/def_function.py", line 840, in _call
return self._stateless_fn(*args, **kwds)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 2829, in call
return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 1848, in _filtered_call
cancellation_manager=cancellation_manager)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 1924, in _call_flat
ctx, args, cancellation_manager=cancellation_manager))
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/function.py", line 550, in call
ctx=ctx)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
inputs, attrs, num_outputs)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 3 root error(s) found.
(0) Invalid argument: Key: joints. Can't parse serialized Example.
[[{{node ParseSingleExample/ParseExample/ParseExampleV2}}]]
[[IteratorGetNext]]
[[StatefulPartitionedCall/hrnet/s3/m0/b11/relu1/Relu/_902]]
(1) Invalid argument: Key: joints. Can't parse serialized Example.
[[{{node ParseSingleExample/ParseExample/ParseExampleV2}}]]
[[IteratorGetNext]]
(2) Invalid argument: Key: joints. Can't parse serialized Example.
[[{{node ParseSingleExample/ParseExample/ParseExampleV2}}]]
[[IteratorGetNext]]
[[ConstantFolding/PartitionedCall/gradients/hrnet/s1/m0/b01/conv3/Conv2D_grad/ShapeN-matshapes-1/_10708]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_step_133797]

Function call stack:
train_step -> train_step -> train_step

In dataset/dataloader.py, I want to know where you referenced the below operation.
I'm just reviewing your code and have been struggling to understand the transformation process. but it is quite difficult to me.
Please let me know what to look for to understand it...

def transform(img, scale, angle, bbox_center, output_shape):
    tx = bbox_center[0] - output_shape[1] * scale / 2
    ty = bbox_center[1] - output_shape[0] * scale / 2

    # for offsetting translations caused by rotation:
    # https://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html
    rx = (1 - tf.cos(angle)) * output_shape[1] * scale / 2 - tf.sin(angle) * output_shape[0] * scale / 2
    ry = tf.sin(angle) * output_shape[1] * scale / 2 + (1 - tf.cos(angle)) * output_shape[0] * scale / 2

    transform = [scale * tf.cos(angle), scale * tf.sin(angle), rx + tx,
                 -scale * tf.sin(angle), scale * tf.cos(angle), ry + ty,
                 0., 0.]

    img = image_ops.transform(tf.expand_dims(img, axis=0),
                              tf.expand_dims(transform, axis=0),
                              fill_mode='constant',
                              output_shape=output_shape[:2])
    img = tf.squeeze(img)

    # transform for keypoints
    alpha = 1 / scale * tf.cos(-angle)
    beta = 1 / scale * tf.sin(-angle)

    rx_xy = (1 - alpha) * bbox_center[0] - beta * bbox_center[1]
    ry_xy = beta * bbox_center[0] + (1 - alpha) * bbox_center[1]

    transform_xy = [[alpha, beta],
                    [-beta, alpha]]

    tx_xy = bbox_center[0] - output_shape[1] / 2
    ty_xy = bbox_center[1] - output_shape[0] / 2

    M = tf.concat([transform_xy, [[rx_xy - tx_xy], [ry_xy - ty_xy]]], axis=1)
    return img, M

Amazing work and very interesting paper.

In terms of using this on other data (such as hand data sets), is it as simple as changing the tf.records file to have those joints?

I see in the coco.py there are some joint defines, but nothing really in the yaml files.

Also, in the coco.py there are some upper body defines. Should these be modified as well? To address rotational bias in the data sets, can the rotational parameters be further modified to include more rotation.

Sorry about all the questions and thank you again for your amazing contribution.

I have error AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'values'.

$ python3 validate.py -c evopose2d_M_f32.yaml Running on CPU Number of accelerators: 1 Traceback (most recent call last): File "validate.py", line 142, in <module> AP = validate(strategy, cfg, split='val') File "validate.py", line 73, in validate scores = np.concatenate(scores.values, axis=0) AttributeError: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'values'

How can i solve this problem?