First-time XR Live Effects Part 2: Music Generation with SunoAI

In the second article in this series, which explains how to create “XR Live,” a live music performance on XR expressed with particles, for beginners, we will explain the part of generating the sound sources to be used with the music generation AI “SunoAI.
This article will follow the flow of “creating images,” “AI generating lyrics,” and “AI generating music based on them.

Flow of generating music with SunoAI

The music generation AI “SunoAI” allows you to enter the mood and tune of a song as a prompt, and it will automatically generate a song based on that.

You can enter lyrics to generate a song with vocals or an instrumental song without vocals, and it also supports Japanese lyrics, so you can easily generate songs with Japanese lyrics automatically to expand your range of expression.

To generate songs with SunoAI, you have to spend credits, and it takes 10 credits to generate a single song. 50 credits are given monthly for the free version, so you can try it for free for up to 5 times after you sign up.

SunoAI-generated music can only be used for commercial purposes with a paid subscription to the Pro Plan or higher.

You can switch between monthly and yearly payments by switching between the “Monthly Billing” and “Yearly Billing” tabs at the top. In this case, I chose monthly payment.

When registering for the first time, select your account information such as Google or Discord, and complete the account verification and payment information to register a subscription.

SunoAI:https://suno.com/

Thinking about the output image

Since our goal is to “generate music for XR Live,” we will start with an image of “what kind of live performance we want to create” before generating music.

This time, we wanted to create an XR Live with “stars” as the main direction from the viewpoint of ease of creation and understanding, so we assumed that the lyrics would be star-themed.
From the image of stars, we connected it to the image of “sparkles” and “excitement” and decided to use the theme of “sparkles of a girl performing live for the first time”.

Basically, based on this image board, we proceed to the creation of XR Live.

Create “Lyrics” in CahtGPT

Before generating music with SunoAI, we prepare lyrics for it. of course, you can think of them yourself, but we will utilize AI. SunoAI also generates music using prompts, and we recommend that you also generate optimized lyrics and prompts for it.

This time, we used a 3D model (Ryu no Yol-chan) of the girl

 If the generated content is OK, you can use it as is, but by going back and forth a few times, you can create lyrics that better suit your image.
I wanted the start of the song to have impact, so I gave the following correction instructions

SunoAI has tags that allow you to set the development of the song and lyrics, but if you are giving instructions in Japanese, it may be “intro” like this

It is easier if you can copy and paste all the text when using SunoAI, so make the changes with instructions at this point.

If this is the case, you can copy and paste as is.

We have instructed them to generate a prompt for SunoAI along with the lyrics, so you should have the following text:

You can copy and paste this area as is, or you can edit it yourself while actually generating the music based on the generated text

.

Generating music

in SunoAI Based on the lyrics and prompts generated by ChatGPT, generate music in SunoAI.

Click on “Create” on the left and you will see a screen like the one on the right, first turn on the “Custom mode” button.

Put the lyrics in Lyrics, the tune and other instructions in Style of Music, and the title of the song in Title. v3″ below that is the version of this AI, so leave it as is (the latest as of May 2024 is v3).
After entering lyrics, tune and title, click the “Create” button to spend 10 credits and start generating music.

The song being generated can be heard even before the generation is complete, although the indicator will be circling for a while after the thumbnail image is generated.

But probably with the current version, if you enter “Japanese Lyrics” + “Japanese Prompt”, the prompt text should be included at the start of the song.
If you start singing the lyrics you specified as is, you may continue.

In this case, I ended up singing “Vocals all over, instruments clear~” in the mix section I had entered in the prompt

To avoid this phenomenon, set the tune prompt to English:
In this case, I set it to English at DeepL, but you can also set this part to English when you first generate the lyrics and prompt in ChatGPT.

Now, you can repeat the generation until you are satisfied with the tune you want to use, and if you find one you like, download it. One point to note is that the songs generated by SunoAI are 2 minutes long, which is fine in this case, but if you want to generate longer songs, you must extend the song by clicking the “Extend” button.

Enter how many minutes and seconds into the original song you want to extend and press the “Extend” button to generate a continuation from that section

Again, as with the song generation, repeat the process until you are satisfied, and when you have generated a continuation that matches your image, click “Get Whole Song” from the menu to complete the extension.

However, even if you extend the song, it will take a lot of patience, as the song does not take over the tune well or the lyrics do not continue well, so at least at this point I would recommend that the song should end within 2 minutes.

For this song, the first chorus ended at about 1 minute 14 seconds, so it would be perfect if the song ends around the end of the chorus.

Just as there are tags like [Verse] and [Chorus], the [end] tag can be used to end a song, so I generated the entire tag with the lyrics by re-directing the above in ChatGPT:

Here are the lyrics generated with that, and when I used this to generate the song, it generated about 1 minute and 30 seconds of music in rapid succession:

Once you have generated asong you like, select Download from the menu and click Audio to download the generated song as an mp3 file.