CRI LipSync in Live2D Cubism 5.0

We are thrilled to announce that Live2D, Inc. has adopted CRI Middleware’s voice analysis and lip-sync technology, CRI LipSync, for Live2D Cubism. Live2D Cubism is the industry standard for creating real-time 2D animations and dynamic character expressions. As such, it is especially popular among virtual YouTubers and game developers.

CRI LipSync uses a unique combination of signal processing and machine learning algorithms to automatically generate natural mouth patterns for animations. Speech is analyzed in real-time and highly accurate lip-sync data is generated without the need to pre-train with data from each speaker.

Demand for lip-sync solutions that accurately synchronize a character’s lips and tongue movements with spoken words has increased in many fields, ranging from video commercials and animations to video games and VTuber platforms. However, until now, creating natural movements that match the voice required considerable time and effort due to the detailed settings needed for lip and tongue positioning. In a real-time scenario, tracking inaccuracies could occur, leading to erroneous character movements.

With CRI LipSync, highly accurate lip-sync can be automatically generated from audio data, significantly reducing both production time and cost.
By including CRI LipSync as a standard feature in Live2D Cubism 5.0, this high-quality lip sync can be applied to characters at any production stage, from the creation of 2D animation models to their real-time usage in applications.

The Live2D Cubism SDK is available for a wide range of platforms, including Windows, macOS (Apple Silicon compatible), iOS, Android, and web browsers. CRI LipSync will support real-time voice analysis on all these platforms.

Moreover, since voice analysis can be performed within the Live2D Cubism Editor itself, creators will be able to check the operation of their models, including lip-sync, when designing it, improving their workflow and speeding up iteration time.

Live2D Cubism 5.0 is available from the Live2D website. A demo can be downloaded for free, and the new sample model “Kei” uses the lip-sync feature.

Do not hesitate to contact us if you want more information about CRI LipSync!