{"id":6046,"date":"2025-06-02T16:43:13","date_gmt":"2025-06-02T07:43:13","guid":{"rendered":"https:\/\/blog.criware.com\/?p=6046"},"modified":"2025-08-06T17:29:17","modified_gmt":"2025-08-06T08:29:17","slug":"sofdec-cue-points-part-2","status":"publish","type":"post","link":"https:\/\/blog.criware.com\/index.php\/2025\/06\/02\/sofdec-cue-points-part-2\/","title":{"rendered":"Sofdec Cue Points [Part 2]"},"content":{"rendered":"<p>In the first part of this tutorial, we exported markers from our After Effects project into a text file. Then, we used this data in the Sofdec Encoding Wizard to embed cue points in our video. In this part, we will explain how to create a Unity script that reacts when the cue points from the .usm file we created are reached during video playback.<\/p>\n<p>To showcase this, we will be using the same scene, representing a virtual fashion show.<\/p>\n<div style=\"max-width: 700px; margin: 0 auto; margin-bottom: 30px;\">\n<div style=\"width: 800px;\" class=\"wp-video\"><video class=\"wp-video-shortcode\" id=\"video-6046-1\" width=\"800\" height=\"450\" preload=\"metadata\" controls=\"controls\"><source type=\"video\/mp4\" src=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/05\/Video.mp4?_=1\" \/><a href=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/05\/Video.mp4\">https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/05\/Video.mp4<\/a><\/video><\/div>\n<\/div>\n<h4><strong>Playback<\/strong><\/h4>\n<p>The full Unity script that starts the video and reacts to the cue points is described below.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Cue-Point-Callback.png\" alt=\"\" width=\"733\" height=\"1296\" class=\"alignnone size-full wp-image-6049\" srcset=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Cue-Point-Callback.png 733w, https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Cue-Point-Callback-170x300.png 170w, https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Cue-Point-Callback-579x1024.png 579w\" sizes=\"auto, (max-width: 733px) 100vw, 733px\" \/><\/p>\n<p>When playing back a video using CRI Sofdec, there is a time lag between the command to start the playback (Start function) and when the first rendering can actually be performed.<br \/>\nTherefore, it is recommended to first call the playback preparation command (PrepareForRendering), which also includes drawing preparation, and then call the Start function, and enable drawing at the same time.<\/p>\n<h4><strong>Callback<\/strong><\/h4>\n<p>By registering a CuePointCallback() function with CriMana (Sofdec\u2019s runtime library), the script will be notified when a specific cue point has been reached during the video playback.  You can find more information about this callback in the official documentation:<br \/>\n<a href=\"https:\/\/game.criware.jp\/manual\/unity_plugin_en\/latest\/contents\/classCriWare_1_1CriMana_1_1Player.html#a52f34523349d55e89bccb13db73f8c98\">https:\/\/game.criware.jp\/manual\/unity_plugin_en\/latest\/contents\/classCriWare_1_1CriMana_1_1Player.html#a52f34523349d55e89bccb13db73f8c98<\/a><\/p>\n<p>In our callback function, the script switches between two light objects and updates the pose of a Humanoid character.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Marker-Playback-Script.png\" alt=\"\" width=\"445\" height=\"132\" class=\"alignnone size-full wp-image-6050\" srcset=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Marker-Playback-Script.png 445w, https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Marker-Playback-Script-300x89.png 300w\" sizes=\"auto, (max-width: 445px) 100vw, 445px\" \/><\/p>\n<p>For our example, we kept the character animation relatively basic. It simply cycles through various poses, one each time \u201cAction\u201d is triggered, as shown in the diagram below. <\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Animator.png\" alt=\"\" width=\"1434\" height=\"433\" class=\"alignnone size-full wp-image-6051\" srcset=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Animator.png 1434w, https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Animator-300x91.png 300w, https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Animator-1024x309.png 1024w, https:\/\/blog.criware.com\/wp-content\/uploads\/2025\/06\/Animator-768x232.png 768w\" sizes=\"auto, (max-width: 1434px) 100vw, 1434px\" \/><\/p>\n<h4><strong>Conclusion<\/strong><\/h4>\n<p>Using cue points to synchronize 3D content or visual effects with a movie can add a whole new dimension to the playback of song videos in Unity, for example.<br \/>\nIf a song and VFX are prepared in the video file, and cue points are set to match the song&#8217;s timing, it becomes possible to synchronize them with the movement of a Humanoid character (including mouth movements, choreography\u2026), and various objects or lights present in the Unity scene.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the first part of this tutorial, we exported markers from our After Effects project into a text file. Then,<\/p>\n","protected":false},"author":2,"featured_media":6048,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"colormag_page_container_layout":"default_layout","colormag_page_sidebar_layout":"default_layout","footnotes":""},"categories":[6,7,23],"tags":[],"class_list":["post-6046","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-sofdec","category-tutorials","category-unity"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/posts\/6046","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/comments?post=6046"}],"version-history":[{"count":4,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/posts\/6046\/revisions"}],"predecessor-version":[{"id":6055,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/posts\/6046\/revisions\/6055"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/media\/6048"}],"wp:attachment":[{"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/media?parent=6046"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/categories?post=6046"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/tags?post=6046"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}