【Unity/Playmaker】How to create an Audio Visualizer

In this article, I will introduce how to create an audio visualizer using Playmaker.
By using this technique, you can create an interactive scene with music.

 

Sample

Unity package

▼You can download from here.
Audio Visualizer.unitypackage

Unity Project

▼You can download from here.
STYLY-Unity-Examples

Please look at the files in STYLY-Unity-Examples/Assets/STYLY_Examples/Audio Visualizer

Introduction

What kind of scene can you create?

By using the Playmaker custom action I introduce in this article, you can create an audio visualizer-like object which will actively react to sounds. For example, you could create an object that change shape according to certain wavelengths, or set it so that an effect will activate if the sound exceeds the threshold. You can use your own ideas to create a more original effect, too.

Preparations

In order to create the audio visualizer I will introduce in this article, you need to install the following 2 softwares.

  • Playmaker (Paid Asset)
  • Playmaker Custom Action (Included in the Unity Plugin for STYLY)

To learn how to install Playmaker, please refer to this article.

To learn how to install Unity Plugin for STYLY, please refer to this article.

※What is a Playmaker custom action?
Playmaker is usually used by combining preset actions to form a logic,
but users can also create new actions to expand Playmaker funcitons.
User created actions are called ‘Custom actions’.

Caution

The audio visualizer looks perfect through a VRHMD, but will not work with a web player.
The reason is because the distributed custom action does not support WebGL.

How to use a custom action

Custom Action

When you open the Playmaker action browser with ‘Playmaker’ and ‘Unity Plugin for STYLY’ installed, you can find custom actions named ‘Init Audio Spectrum’ and ‘Get Audio Spectrum Level’ in the STYLY item.

By using these custom actions, you can analyze the audio spectrum of the music in the scene, and access those values.

Init Audio Spectrum:
This will set the parameters to analyze the audio spectrum.

Get Audio Spectrum Level:
This will get the value of the audio spectrum of a specific wavelength

How to use a custom action

I will introduce how to analyze the audio spectrum of the music in the scene, and how to access those values.

You will need an object in the scene in order to execute ‘Init Audio Spectrum’.
If you use ‘Get Audio Spectrum Level’ and specify the Index, you can store certain wavelength values as Variables. By using these Variables and combining them with other Playmaker actions, you can create many uses.

For example, if you want to create an audio visualizer, you can use the ‘Set Scale’ action and use the variables from ‘Get Audio Spectrum Level’ to set the scale of the object.

Custom Action settings

Init Audio Spectrum

Item Description
Number Of Samples The number of samples you gain from the spectrum. The number that fills xxx on Sampling_xxx will be the number of samples. The larger the number, the higher the accuracy, but the performance will become worse.
Frequency Band Type The type of the frequency band. It is mainly the number of the frequency bands. If you use ‘Ten Band’, the spectrum will become divided into 10 bands.
Fall Speed The falling speed of the spectrum peak value.
Sensibility The sensibility of the spectrum. If this value is high, it will transform quicker.

Get Audio Spectrum Level

Item Description
Index This will determine which spectrum it will analyze, counting the frequency from the bottom. If you want to take the first (lowest) frequency, you should set this to 0.
Bar Type This determines how you take the spectrum values. If you plan nothing special, you are probably fine with ‘Mean Level’.
・Realtime:The realtime spectrum value
・Peak Level:The peak value
・Mean Level:The mean value. This is usually the smoothest.
Multiplier The value to multiply the spectrum value. The result will be the value for Store Value, below.
Store Value The final spectrum value. If you want to use it for something, you need to store it into a variable.

How to create an audio visualizer

I will explain how to create an audio visualizer using the custom action.

Add an object with ‘Init Audio Spectrum’ in the scene

First, add an object with ‘Init Audio Spectrum’ to the scene.

Right click within the hierarchy window, and choose ‘Create Empty’ to create an object. Name it so you can recognize it afterwards. I named it ‘AudioSpectrumInitializer’ for this tutorial.
Right click on the Playmaker Editor while selecting the ‘AudioSpectrumInitializer’, and select ‘Add FSM’.

Open the Playmaker action browser, find the STYLY item, and double click on the ‘Init Audio Spectrum’ custom action. Now, the action has been added to the State.

Use ‘Get Audio Spectrum Level’ and change the object size according to the music

Next, create an object that will change size according to sound.

First, create an object. In this tutorial, I will use a Cube that will change size according to sound, and create a simple audio visualizer.
Right click on the hierarchy window, and select ‘3D Object → Cube’ and place a Cube in the scene. I named it ‘AudioVisualizeCube’.
While selecting the ‘AudioVisualizeCube’, right click on the Playmaker Editor, and select ‘Add FSM’.

Open the Playmaker action browser, find the STYLY item, and add the ‘Get Audio Spectrum Level’ custom action to the State. Also, add the ‘Set Scale’ action from ‘Transform’ too.

Lastly, we need to edit the settings of the custom action.

Index
The ‘Frequency Band Type’ of ‘Init Audio Spectrum’ is ‘Ten Band’, so we need to set it between 0 – 9. I used 0 for my example.

Bar Type
The ‘Realtime’, ‘Peak Level’, and ’Mean Level’ changes the movements. Select Mean Level here.

Multiplier
This value will be multiplied to the result. The default value is 100.

Store Values
This specifies the Variables to store the values in.
Click on the 2 line button on the right, and change it so it can specify a variable. From the pull tab, select ‘New Variables…’, add a new variable named ‘level’, and store it there.

Set Scale
We will create a audio visualizer that only moves in the Y-axis direction.
Use the ‘level’ variable we just created for ‘Y’.
Also, check the ‘Every Frame’ box so that it will react to the sound for every frame.

Place a sound source

The audio visualizer will not move unless there is a sound source, so we need to place one in the scene.

Right click in the hierarchy window, choose ‘Audio → Audio Source’ and place it in the scene.
Open the Audio Source Inspector window, and attach a sound file to the Audio Clip.

Done!

Now we have completed the audio visualizer.
Click the Unity Play button, and see if the visualizer works properly.
If you want to create a scene where multiple cubes move according to certain frequencies, create more and set the ‘Get Audio Spectrum Level’s Index’ to 0 – 9, and place them accordingly.

Upload to STYLY

Create a Prefab from the AudioSpectrumInitializer and audio visualizer object, and upload to STYLY. You can store these 2 in a single empty object, and upload them as one.

To learn how to upload assets from Unity to STYLY, please refer to the following article.

 

In the next article, I will introduce how to use the audio visualizer to create a simple VR music promotion video.