How to use other existing model ? about ofxtensorflow2 HOT 11 CLOSED

hey stephanschulz,

happy to hear that you enjoy it :)
A key feature of this addon is that it eases the inference of any TensorFlow 2 SavedModel. Such a model may be:

• incluced in the repository your are aiming at
• created from a weight file and the code to contstruct the model
  • for .h5 files: see keyword spotting example
  • for check-points: see style transfer example
• saved after/during running the training: see pix2pix example training step

if I stumble accross a TensorFlow2 implementation i ll let you know.

from ofxtensorflow2.

@bytosaur thanks for the reply.

i realize people certainly have different levels of working with TensorFlow; people that can and want to train their own models and beginners like me that just want to run existing model.
If i understand you correctly, if i want to use the handpose model i saw in the JS example i need to first find a TensorFlow2 implementations of it?
I will keep at it and try to learn more :)
Cheers.

from ofxtensorflow2.

this might be what you are looking for https://github.com/MikeOfZen/Yet-Another-Openpose-Implementation
it s a TF2 implementation with pretrained models.

looking forward to see what you can do with it! Let us know as we can not have too less examples here :)

from ofxtensorflow2.

Thanks for finding this.
It does indeed have the saved_model.pd and variables files. https://github.com/MikeOfZen/Yet-Another-Openpose-Implementation/tree/master/trained_models/model11_test-15Sun1219-2101

Now i just have to figure out how to send me images / video stream to the model.
Maybe the pix2pix example is a good start since it uses input = ofxTF2::imageToTensor(imgIn); and ofxTF2::tensorToImage(output, imgOut);

from ofxtensorflow2.

make sure to check out the READMEs in the examples. Especially example_basics gives a good overview of how to use this addon.
Afterwards, check both example_pix2pix and example_style_transfer as they are dealing with image input. It's also a good way to learn how to use the ThreadedModel. Altough for test purposes it is not required.

Are you sure this model outputs an image? i would expect it to output a vector of joints...

from ofxtensorflow2.

You are right it does not output an image but a long vector with about 139000 entries.

The repo you found has a Skeletonizer class which seems take the model output and converts it to the skeleton points, i guess ?

        pafs, kpts = self.model.predict(input_img)
        pafs = pafs[0]
        kpts = kpts[0]
        skeletonizer = post.Skeletonizer(kpts, pafs)

I believe the next step would be to translate this Skeletonizer to c++.
https://github.com/MikeOfZen/Yet-Another-Openpose-Implementation/blob/dbbb558bc95b15dfa3976514dad2c7803599f5c3/post_processing/post.py#L206

from ofxtensorflow2.

Yes, pre and postprocessing are important steps to tackle. Overriding the runModel() function of a ofxTF2::Model is the recommended way to insert these steps.

For some codes, even operations that are not-related to AI are embedded into the SavedModel. The authors of the pyhton code for example_keyword_spotting use kapre. It's a pyhton package that formulates the computation of e.g spectrograms as Tensorflow graphs. This way we dont have to care about it when running the model :)

from ofxtensorflow2.

thanks again.
this is a lot of new information for my brain. i will see if i can make sense of it all.

from ofxtensorflow2.

happy to help :)

btw, i also found these implementations:
https://github.com/tensorlayer/hyperpose
https://github.com/michalfaber/tensorflow_Realtime_Multi-Person_Pose_Estimation

from ofxtensorflow2.

i will check them out. maybe they are easier to extract keypoints from.
i should mention that i am actually looking for handpose not bodypose detection ;)

from ofxtensorflow2.

Thanks again for providing this information.
As of now i have not managed to make this work. Will keep on trying !
But if you happen to have time to add handpose and bodypose examples i am sure me and other people would be very happy ;)
All the best.

from ofxtensorflow2.