{"id":4271,"date":"2023-04-07T10:05:44","date_gmt":"2023-04-07T01:05:44","guid":{"rendered":"https:\/\/blog.criware.com\/?p=4271"},"modified":"2024-05-20T15:07:47","modified_gmt":"2024-05-20T06:07:47","slug":"using-adx-lipsync-in-unity","status":"publish","type":"post","link":"https:\/\/blog.criware.com\/index.php\/2023\/04\/07\/using-adx-lipsync-in-unity\/","title":{"rendered":"Using ADX LipSync in Unity"},"content":{"rendered":"

In our previous post<\/a> we described the offline generation of mouth patterns by analyzing speech data with CriLipsMake, and how artists can use them to create facial animation in their tools.<\/p>\n

This time, we will see how such mouth patterns can also be obtained by analyzing audio played in real-time, by using the ADX LipSync Library in Unity.<\/p>\n

Unity Plug-in<\/h2>\n

The easiest way to use ADX LipSync in Unity is via the extension package provided with the CRIWARE Unity Plugin. Its location is:<\/p>\n

\\expansion\\cri_adx_lipsync\\unity\\plugin\\cri_adxlipsync_unity_plugin.unitypackage.<\/p>\n

The plugin operation is simple and consists in connecting a sound source \u2013 via a CriAtomSource<\/em> \u2013 to a dedicated CriLipsDeformer<\/em> component. This component analyzes the sound in real-time and controls the mouth movements of a character during audio playback.<\/p>\n

\"01<\/p>\n

Character model preparation<\/h2>\n

The character model has to be prepared in advance in order for its mouth shape to be controllable from the ADX LipSync data. Blend shapes created in an external tool such as Maya or Blender can be used, as well as those made directly with the Unity Animator functions.<\/p>\n

Once your model is ready, it can be imported directly into your scene. For instance, the character model used for this post has nine blend shapes corresponding to the following two mouth-shaped morph targets generated by ADX LipSync:<\/p>\n