microsoft / microsoft-rocketbox Goto Github PK

I'm having trouble importing models in Blender. I know this dataset is meant for Unity, but I still hope for some help with Blender specifically.

Here is the script that I eventually came up with to import a single FBX file:

from pathlib import Path
import bpy

rocketbox_root = Path('/path/to/cloned/Microsoft-Rocketbox')
avatar_path = rocketbox_root / 'Assets/Avatars/Adults/Female_Adult_01/Export/Female_Adult_01.fbx'

bpy.ops.object.delete(use_global=False, confirm=False)  # delete cube
bpy.ops.import_scene.fbx(filepath=str(avatar_path))

# Fix paths to textures
for im in bpy.data.images:
    im.filepath = im.filepath.replace('../../../../../temp/Humans/with_opacity_version/f001/textures', '../Textures')

# Workaround: make opacity texture fully transparent
bpy.data.materials['f001_opacity'].blend_method = 'CLIP'
bpy.data.materials['f001_opacity'].alpha_threshold = 1

In order to use it, launch Blender, switch to the "Scripting" tab, create a new script via the "+ New" button, paste this code and change the rocketbox_root variable to the path of the cloned repository.

In this script, I replace paths to textures which are hardcoded in the FBX file. In addition, I disable the opacity texture by making all polygons associated with it fully transparent. Here is what the imported model looks like without that step:

If I set blend method to "alpha blend", I get this:

The result with my workaround (mode = alpha clip, clip threshold = 1) is as follows:

All of these images are rendered with the Eevee renderer, on white background. I am using Blender 2.83.

I am also not sure what the purpose is for the eyelash opacity textures. The eyelashes are already baked into the color map for the head texture, why duplicate them in the opacity texture?

Are there any guidelines for importing the models in Blender? Ideally, I would prefer a Python script that does everything, not instructions on what things need to be changed in the GUI.

Rocketbox used to come with facial animations before it was open sourced (referenced in posts dating back to 2014).

"m_male_face_expressions" (and presumably a female equivalent).

These animations are missing from the repository.

Can they be added?

Not really a issue but no we're else to ask really. But wondering if anyone has successfully retargeted to ue4 maniquin skelaton

• https://github.com/microsoft/Microsoft-Rocketbox/blob/8046857/Assets/Avatars/Adults/Female_Adult_10/Export/Female_Adult_01.fbx should be Female_Adult_10.fbx.
• https://github.com/microsoft/Microsoft-Rocketbox/blob/8046857/Assets/Avatars/Professions/Security_Male_02/Export/Security_Male_01.fbx should be Security_Male_02.fbx

I'm not a Unity expert so perhaps there's something I'm doing wrong when importing? The CMU mocap database animations (http://mocap.cs.cmu.edu/) don't seem compatible with these models and the results are somewhat terrifying! It would be really useful to upload a small unity project with 3/4 of these models using a couple of open source animations (e.g. Mocap from CMU & mixamo) to promote research use. I'm using Unity 2018.4.12.

Reposting here the link to an issue I opened in the MoveBox repository:
microsoft/MoveBox-for-Microsoft-Rocketbox#8 (comment)

It is difficult to use this as an agent character because avatars do not have a blend shape for blink.
In addition, this rig configuration does not match the humanoid rig. Then, I cannot apply humanoid animations to avatars

I'm having this issue - I placed the .cs file under Assets/Editor and edited it from Generic to Humanoid. This allowed me to set the avatar as humanoid, however it has a bunch of Max related errors and the hair and textures are not working correctly.

I'm not a Unity expert so perhaps there's something I'm doing wrong when importing? The CMU mocap database animations (http://mocap.cs.cmu.edu/) don't seem compatible with these models and the results are somewhat terrifying! It would be really useful to upload a small unity project with 3/4 of these models using a couple of open source animations (e.g. Mocap from CMU & mixamo) to promote research use. I'm using Unity 2018.4.12.

Hello,

Thank you for providing this great package! I just wanted to let you know that it might be helpful to mention in the documentation / read.me that the attached script "FixRocketboxMaxImport.cs" overwrites the ModelImporterAnimationType (L62). This leads to the fact that you cannot manually select "Humanoid" in the ImportSettings of an avatar.fbx in Unity. A brief note on this would certainly have saved me some time and prevented some confusion.

Many thanks and best regards
David

Is there a tutorial, guide, or script to make these models Unity-ready, without having to fix hundreds of bones hierarchies? Or is there a set available that is Unity-ready?

The RocketBox paper mentions that the avatars have 53 body bones and 28 facial bones. Where can I find the list of the names of all the bones and a figure to map these bone-names to their location?
I require this to map the pose data from a video to the RocketBox avatars. I am aware the MoveBox repository provides this functionality for the VIBE pose detection algorithm.
I am planning on using a different pose detection method (ex: MediaPipe Pose) and hence, would need the skeleton structure of RocketBox avatars.
Thanks.

https://github.com/microsoft/Microsoft-Rocketbox/blob/8046857/Assets/Avatars/Adults/Male_Adult_18/Textures/m023_head_normal.tga is not a valid TGA file:

> file Assets/Avatars/Adults/Male_Adult_18/Textures/m023_head_normal.tga
Assets/Avatars/Adults/Male_Adult_18/Textures/m023_head_normal.tga: data

In contrast, other .tga files are valid:

> file Assets/Avatars/Adults/Male_Adult_18/Textures/m023_body_normal.tga 
Assets/Avatars/Adults/Male_Adult_18/Textures/m023_body_normal.tga: Targa image data - RGB 2048 x 2048 x 24

Hello!

Is it possible to implement live lipsync on these models using just the bones in Unity? I saw that the models don't have blendshapes at all so I was curious if this was possible.

Regards

I follow the instructions via https://youtu.be/gqJTjuyNfh8 and this works.

Then I export the model and animation as the .fbx format. However, when I import .fbx into the Blender, the mesh is distorted. Could anyone help me with this?

Hi,

I'm very interested in using some of your avatars for my research project. I'd like to use them in Unity and Vicon Pegasus, however, within Vicon Pegasus the characters are not in a T-pose which causes issues when I want to align the Vicon objects with the avatar (see 2 photos: Imported avatar in Pegasus). The avatar's arms and shoulders are all crushed when I stream the data in Unity as I had to change the position of the avatar in Pegasus to better align the avatar towards the Vicon objects (see photo: Avatar in Unity).

Would it be possible to change the skeleton of the avatars in T-pose by default? So I can import them in Pegasus and Unity to avoid this issue. I've tried it myself by changing the skeleton in Maya before importing them in Pegasus and use that also in Unity but that didn't solve the issue.

Appreciate you're help

Many thanks,
Iris

All models except for Female_Adult_16 and Chef_Female_01 have a material with a name following the pattern <1-2 chars><3 digits>_head. Female_Adult_16 does not have a head material at all, presumably because the model is quite special.

In fact, Female_Adult_16 has only one material, named f204.

In contrast, Chef_Female_01 does have a head:

This model has two materials: f110_body and f110_A_head. Perhaps it would be better to rename f110_A_head to f110_head for consistency?

Many thanks @margonzalezfranco for this initiative !

Some years ago our research on crowd simulation and crowd rendering was based on the first generation of this dataset.

https://www.youtube.com/watch?v=xE7tK7wWinM

It was great to know Microsoft has open source this library.

Many of the animals are mising from the source folders as i see. There i see only the image of the model but not the .max file.
If can update it will be great. Thank you

I've tried downloading these all as a zip but it appears to be corrupted.

Every time I try to extract the contents of the zip, I get an error saying the archive is corrupt.

I am also getting lots of errors when trying to download individual files instead; tga files that won't open, fbx files that appear to be corrupted.