ADX & Unity Synchronization [Part 1]

We recently learned how to synchronize a Unity scene with the content of a video played back by the Sofdec run-time, using the cue points feature. In this 2-part tutorial, we will see how we can synchronize a scene with the audio played back by ADX, to make it more realistic or more immersive.

In the scene depicted above, 2 synchronization mechanisms are used when the police knock on the door.

1 – Synchronization of a character’s voice and subtitles
The random dialog line for the policeman is played every 3 seconds, and the corresponding subtitles are displayed. When the voice ends, the subtitles automatically disappear from the screen.

2 – Shaking based on audio events
Markers are placed in the Cue that implements the knocks on the door, and a callback is registered. The code will receive a notification during audio playback when the markers are reached, triggering the movement of the camera and the door.

In this post, we will focus on synchronizing the character’s voice with the subtitles, and in the next one, we will implement the shaking based on the audio events.
To synchronize the voice with the subtitles:

1 – Create a Cue for each dialog line in AtomCraft and export the ACB file to Unity.
(The voice we used in the sample project was generated using this tool: https://ondoku3.com/ja/)

2 – Add the VoicePlayer (described below) script to an empty object, and assign multiple CRI Atom Sources to other objects, with the corresponding text in array format.

3 – Check the key points of the VoicePlayer script

For voice and subtitles synchronization:
We monitor the playback status (source.status) of the CriAtomSource and hide the subtitles when the voice ends.

For random playback:
We randomly select a dialog line using Random.Range(0, atomSources.Length).

For subtitles display:
If the previous subtitleCoroutine is running, we stop it (StopCoroutine) and execute ShowSubtitle().

In addition to the method described this time, it is possible to store the dialog text and duration in AtomCraft markers and have Unity receive and display that information. However, for this first post, we deliberately introduced a simple script that is easy to implement for developers familiar with Unity, but not necessarily with ADX.

Next time, we will use the Markers to shake the door and camera according to the timing of the door knocks.