{"id":2503,"date":"2019-09-19T13:02:28","date_gmt":"2019-09-19T04:02:28","guid":{"rendered":"https:\/\/blog.criware.com\/?p=2503"},"modified":"2019-09-19T13:23:59","modified_gmt":"2019-09-19T04:23:59","slug":"controlling-particle-effects-with-audio-input-with-unity-adx2","status":"publish","type":"post","link":"https:\/\/blog.criware.com\/index.php\/2019\/09\/19\/controlling-particle-effects-with-audio-input-with-unity-adx2\/","title":{"rendered":"Controlling Particle Effects with Audio Input with Unity &#038; ADX2"},"content":{"rendered":"<h2 style=\"font-size: 150%; font-weight: bold;\">Introduction<\/h2>\n<p>Sound in games is often a reactive one-way system, where the player\u2019s interactions with the world result in sounds being triggered. ADX2 allows the designer to push the boundaries by offering some powerful audio input functionality. This means we can turn the tables by creating sounds which control the world instead. In this month\u2019s blog, we will look at an approach for manipulating particles in Unity using a microphone.<\/p>\n<div style=\"max-width: 800px; margin: 0 auto; margin-bottom: 20px;\">\n<div style=\"width: 800px;\" class=\"wp-video\"><video class=\"wp-video-shortcode\" id=\"video-2503-1\" width=\"800\" height=\"450\" preload=\"metadata\" controls=\"controls\"><source type=\"video\/mp4\" src=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/Particles.mp4?_=1\" \/><a href=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/Particles.mp4\">https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/Particles.mp4<\/a><\/video><\/div>\n<\/div>\n<p style=\"\n    border: 1px solid #ccc;\n    text-align: center;\n    padding: 10px;\n\"><br \/>\n<a href=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/MicrophoneInput.cs_.zip\"rel=\"\">MicrophoneInput.cs<\/a><br \/>\n<em>This project was created using CRIWARE SDK for Unity V2.99.00 and Unity V2019.1.6f1<\/em>\n<\/p>\n<p>&nbsp;<\/p>\n<h2 style=\"font-size: 150%; font-weight: bold;\">Particle Setup<\/h2>\n<p>We can control any type of particle created by the Particle System by enabling the \u201cExternal Forces\u201d setting in the Inspector. This will result in our particle system being affected by Wind Zones, which is a handy feature that allows us to apply a directional force to our particles.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/15-1.png\" alt=\"15-1\" width=\"632\" height=\"147\" class=\"alignnone size-full wp-image-2507\" srcset=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/15-1.png 632w, https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/15-1-300x70.png 300w\" sizes=\"auto, (max-width: 632px) 100vw, 632px\" \/><\/p>\n<p>Next, we need to create our Wind Zone. In my example, I have attached the Wind Zone to my player controller. By doing this, the directionality of the smoke can easily be controlled by approaching the particle source from different angles. If the Wind Zone is selected during gameplay, you will be able to see the directionality of the Wind Zone (as in the video example), as well as the force applied via the Main attribute. This is all we will need to get started with particle control.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/15-2.png\" alt=\"15-2\" width=\"469\" height=\"165\" class=\"alignnone size-full wp-image-2521\" srcset=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/15-2.png 469w, https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/15-2-300x106.png 300w\" sizes=\"auto, (max-width: 469px) 100vw, 469px\" \/><\/p>\n<p>&nbsp;<\/p>\n<h2 style=\"font-size: 150%; font-weight: bold;\">Microphone Input<\/h2>\n<p>Unlike in previous months\u2019 blogs, we won\u2019t need to build any audio data in Atom Craft. Instead, we can simply load the CRIWARE plugin package into Unity to make use of its microphone API.<\/p>\n<ol>\n<li>First, let\u2019s select GameObject \u2192 Create Empty, and name it \u201cMicrophone Handler\u201d<\/li>\n<li>We can then drag our new object into our scene<\/li>\n<li>Select Add Component \u2192 New Script in the Inspector, and call it \u201cMicrophoneInput\u201d<\/li>\n<\/ol>\n<p>To manage the bulk of the microphone input, we can copy from the <em>InputCapture.cs<\/em> file included in the <em>ScriptSample07_InputCapture<\/em> project that comes with the CRIWARE SDK for Unity. This script provides some handy boilerplate code that handles a lot of safeguarding from errors, such as trying to access a microphone that doesn\u2019t exist etc.<\/p>\n<p>I have kept the script largely the same, only removing the code related to drawing the waveform with the Line Renderer and adding in the code necessary to control the Wind Zone (see the script attached to the start of this blog).<br \/>\n&nbsp;<\/p>\n<h2 style=\"font-size: 150%; font-weight: bold;\">Variables<\/h2>\n<pre>\r\n<code class=\"hljs\">\r\n    CriAtomExMic mic;\r\n    bool isRecording;\r\n    float[] micdata = new float[512];\r\n\r\n    public WindZone wind;\r\n<\/code>\r\n<\/pre>\n<p>The global variables defined at the top offer a good overview of what is being achieved, and how the edited script differs from the example one.<\/p>\n<ol>\n<li>First, we create the variable \u201cmic\u201d to access the <em>CriAtomExMic<\/em> class<\/li>\n<li>Next, we create the Boolean \u201cisRecording\u201d to keep track of our recording state<\/li>\n<li>Then, we create the buffer \u201cmicdata\u201d for our microphone data and define its size<\/li>\n<li>And finally, we create a public variable for our Wind Zone, where it can then be attached to our script in the Inspector<\/li>\n<\/ol>\n<p><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/15-3-1.png\" alt=\"15-3\" width=\"640\" height=\"183\" class=\"alignnone size-full wp-image-2509\" srcset=\"https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/15-3-1.png 640w, https:\/\/blog.criware.com\/wp-content\/uploads\/2019\/09\/15-3-1-300x86.png 300w\" sizes=\"auto, (max-width: 640px) 100vw, 640px\" \/><br \/>\n&nbsp;<\/p>\n<h2 style=\"font-size: 150%; font-weight: bold;\">Update<\/h2>\n<pre>\r\n<code class=\"hljs\">\r\n    void Update()\r\n    {\r\n        if (isRecording)\r\n        {\r\n            if (Input.GetKeyUp(KeyCode.F)) {\r\n                wind.windMain = 0;\r\n                StopMic();\r\n            }\r\n        }\r\n        else\r\n        {\r\n            if (Input.GetKeyDown(KeyCode.F)) {\r\n                StartMic();\r\n            }\r\n        }\r\n\r\n        if (mic == null) {\r\n            return;\r\n        }\r\n\r\n        while (true) {\r\n            \/\/ Get audio input data\r\n            uint numSamples = mic.ReadData(micdata);\r\n            if (numSamples == 0) {\r\n                break;\r\n            }\r\n        }\r\n\r\n        \/\/ Control the strength of the wind with microphone input data\r\n        for (int i = 0; i < micdata.Length; i++) {\r\n            wind.windMain = Mathf.Abs(micdata[i] * 250);\r\n        }\r\n    }\r\n\r\n<\/code>\r\n<\/pre>\n<p>The revised update method is straight-forward. Instead of using a separate UI overlay as in the example, we are going to control the on-off state of the microphone using the \u201cF\u201d key. If the \u201cF\u201d is pressed, we call the <em>StartMic<\/em> function defined by the example code, which handles creating and assigning the microphone. If the \u201cF\u201d is released, we call the <em>StopMic <\/em>function, which stops and disposes of the microphone instance, freeing up the system resources.<\/p>\n<p>The \u201c<em>if (mic == null)<\/em>\u201d works as a failsafe, preventing weird potential behaviour from occurring as a result of trying to access some data if no microphone has been detected. If there is a microphone, the script continues to read audio data into our buffer.<\/p>\n<p>The last part of our update method is where the fun happens! Here we are looping through our buffer, controlling the strength of our wind via <em>windMain<\/em> with the scaled value of our microphone input. The absolute (Mathf.Abs) value of our input data is taken since these values will otherwise oscillate between positive and negative values (which would result in our wind force periodically reversing).<br \/>\n&nbsp;<\/p>\n<h2 style=\"font-size: 150%; font-weight: bold;\">Conclusion<\/h2>\n<p>Of course, this system could use a lot of tweaking to ensure, for example, particles are only affected if they are within a certain distance of the player. However, even with a conservative amount of code, we can already start to flesh out some sophisticated input-driven control! This can be especially handy for VR, as headsets usually have microphones integrated into them, and interacting with the world in this way can significantly enhance immersion.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction Sound in games is often a reactive one-way system, where the player\u2019s interactions with the world result in sounds<\/p>\n","protected":false},"author":2,"featured_media":2525,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"colormag_page_container_layout":"default_layout","colormag_page_sidebar_layout":"default_layout","footnotes":""},"categories":[5,7],"tags":[],"class_list":["post-2503","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-adx","category-tutorials"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/posts\/2503","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/comments?post=2503"}],"version-history":[{"count":14,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/posts\/2503\/revisions"}],"predecessor-version":[{"id":2523,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/posts\/2503\/revisions\/2523"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/media\/2525"}],"wp:attachment":[{"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/media?parent=2503"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/categories?post=2503"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.criware.com\/index.php\/wp-json\/wp\/v2\/tags?post=2503"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}