I was recently approached by someone working on a project with an interesting problem. They were capturing facial animation via FaceCap but wanted to apply this motion capture to a Mixamo model. But a common problem is how to get the blend shapes and hierarchy from System X to System Y. In this case, FaceCap outputs a model and shapes with a specifically named hierarchy, while your model most likely does not have the same hierarchy. Especially if it is something you bought off Mixamo.
So how do you efficiently convert between these systems? This was the problem Baroquedub had. And considering I’ve done similar things in the past, he asked if I could help. So we created this Animation Retargeting tool to allow for him to easily convert his clips.
As you can see there are a number of parameters that need to be modified and doing so by hand would be problematic. But the end result is worth it! By blending the facial capture with the Mixamo models, baroquedub was able to get very good results.
So how does this generally work?
- Create your facial mocap using FaceCap and import it into Unity.
- Import the model you want to use your mocap with, hopefully with a similar rig setup.
- Add the animation clip as a layer blended on top of the existing animations (so the model can run/walk/etc)
- Add the Retargeting utility to Unity via the Package Manager (instructions on Github) and open the window.
- Add the clip, folder, or animator to the list of items to process.
- Choose the preset you’re interested in, copy a preset, or create a new one to match your rig.
- Press ‘Retarget’ and your clips should now be correctly formatted for your rig!
I really liked baroquedub’s workflow and experimentation, and I have to give him all the credit for this. I just wrote some editor code for him, and felt it would be helpful to the internet at large. I am interested in any feedback you may have about it, and any improvements that could be made. I have attempted to make it general enough that it can be applied to a number of animation problems.
Here is a longer form video going over a lot of the finer points. Check out baroquedub’s channel for more cool videos!