Keeping SFX in Key

Introduction

When designing sounds for our game, we might decide to use pitched musical sound effects such as pizzicato strings. This approach can add stylistic charm to our soundscape, which can help unify the score and sound design. However, depending on how we compose our music, it might be that we trigger sound effects which don’t fit in the key of the music. Thankfully, ADX2 can easily re-pitch audio, and by embedding markers in our score, we can even automatically inform ADX2 of the current key.

This project was created using CRIWARE SDK for Unity V3.00.04 and Unity V2018.4.19f1

DAW

Once we are finished composing our music, we can embed markers into our waveform data using our DAW. Using the Template provided (which can be found in View→Sequence Marker Auto Generation Setting…), we can create markers and name them, for example:

CALLBACK NAME=Cmaj ID=0 TAG=Cmaj

We can create as many of these as we have keys and key changes, setting the TAG accordingly. The NAME attribute won’t affect our callback system, but we can name it the same as our TAG to keep track of it in Atom Craft. We can also create loop markers around our music using LOOPSTART and LOOPEND, respectively.

Atom Craft

Before proceeding, we need to enable Auto Marker Generation. To do this navigate to View -> Sequence Marker Auto Generation Setting for Waveform Registration and tick the box at the top of the window.
By following the above, dropping our music Waveform Material onto our CueSheet should give us all we need for our music system. We just need to ensure that the relevant Waveform Material has Override Loop Information Flag set to True, and Loop Type set to One Shot (since looping will be controlled by the embedded Loop Markers instead).

Next up, we have the main feature of our system; the pitched sound effects. All we need is a Cue for each of our musical keys, and several instances of the same sound effect. We can then change the pitches of our sounds to cover whatever range or configuration we want. For testing, we have decided to use a Combo Cue which covers a whole octave. Starting with the root note, we can pitch our sounds down in cents to match the notes of the scale.

With our music and pitched sound effect Cues set up, we can build our project for Unity.

Unity

Once we have ADX2 set up and our project loaded into Unity, we can create a callback script to check what key we’re in and change the name of the Cue being referenced. As in a previous blog, we can load the following script into our project to make handling Callback information simpler, which can be implemented by adding the using directive at the top of our code:

https://gist.github.com/TakaakiIchijo/bde623ce973c64b5ed11da3160f3c370

For testing purposes, I have created a Cube object and attached a CriAtomSource component, as well as the following script:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using CriWareExtensions;

public class CueKeySwitcher : MonoBehaviour {

    CriAtomSource source;

    void Start() {
        source = GetComponent();

        CriAtomExSequencerExtension.SetEventCallback((e) => {
            switch (e.eventTag) {
                case "Cmaj": ChangeKey ("Cmaj");
                    break;
                case "Cmin": ChangeKey ("Cmin");
                    break;
                default:
                    Debug.Log("Current Key is Unknown");
                    break;
            }
        });
    }

    private void OnMouseDown() {
        source.Play();
        Debug.Log("Playing from Cue: " + source.cueName);

        // Changes cube colour randomly on click
        GetComponent().material.color = Random.ColorHSV();
    }

    void ChangeKey(string currentKey) {
        source.cueName = currentKey;
        Debug.Log("Music is currently in Key: " + currentKey);
    }
}

The ChangeKey method receives a string which is used to reference/select the correct Cue. Debugging here is helpful to ensure that the callback has been successfully read and updated.
The Start method is first setting source to the attached CriAtomSource component, and then calling CriAtomExSequencerExtension.SetEventCallback to check the Tag of our Callback marker when it is registered. Using a case statement, we can call our ChangeKey method and set the Cue to play notes from the correct respective Cue.
Finally, the OnMouseDown method is playing our sound back each time the cube is clicked, and it is changing the color of the cube for visual feedback (coupled with a debug statement to check that the correct Cue is playing).