Google I/O 2023: MusicLM And Text To Video Show With Dan Deacon and Phenaki and Bard AI Tools

[FurkanGozukara]

Google I/O 2023: MusicLM And Text To Video Show With Dan Deacon and Phenaki and Bard AI Tools

Full tutorial: https://www.youtube.com/watch?v=uTE0diuotmw

The video features musician Dan Deacon performing live on stage with Gmail and Google Calendar as his all-time favorite collaborators. He introduces the use of AI tools like Fnacke, Bard, and MusicLM in creating his music. The first song he performs is called "When I Was Done Dying," and the accompanying video is generated using the song's lyrics. Dan Deacon then leads the audience in a guided visualization, imagining themselves floating above the Shoreline Amphitheater and encountering a bird with lips, which invites them to be roommates and create a community. He discusses his fascination with advancements in musical technology throughout history and his excitement to work with AI researchers at Google, exploring new ways of making sounds and music. He mentions using AI tools like text to music, Sing Song, and others to generate music and visuals for the performance. Dan Deacon introduces his band, Chiptune, and shares how they created a song using AI tools by inputting the word "Chiptune." He describes the different sections and elements of the song they created, including a breakdown and a full release. He concludes by returning to the guided visualization and expressing a desire to manifest a duck with lips, symbolizing a utopian life and a unique roommate experience. He performs his final song called "Change Your Life" and thanks the audience, mentioning that he will be doing a five-hour set later. The visuals accompanying the performance are generated using Pinaki.

Source Google. Full Event (10 May 2023) ⤵️

https://www.youtube.com/watch?v=cNfINi5CNbY

Our Discord server ⤵️

https://bit.ly/SECoursesDiscord

If I have been of assistance to you and you would like to show your support for my work, please consider becoming a patron on 🥰 ⤵️

https://www.patreon.com/SECourses

Technology & Science: News, Tips, Tutorials, Tricks, Best Applications, Guides, Reviews ⤵️

https://www.youtube.com/playlist?list=PL_pbwdIyffsnkay6X91BWb9rrfLATUMr3

Playlist of StableDiffusion Tutorials, Automatic1111 and Google Colab Guides, DreamBooth, Textual Inversion / Embedding, LoRA, AI Upscaling, Pix2Pix, Img2Img ⤵️

https://www.youtube.com/playlist?list=PL_pbwdIyffsmclLl0O144nQRnezKlNdx3

#Google #GoogleIO #MusicLM

In this unique and engaging video, Dan Deacon, a renowned musical composer, takes you on a journey of technological advancements in music and his own musical explorations. An exploration of sound, technology, and artificial intelligence is embarked upon, with a particular focus on MusicLM, Fnacke, and Bard. The creation of a song, "When I Was Done Dying," using these innovative technologies is also shared.

In a whimsical twist, Dan leads a guided visualization in which viewers are invited to imagine themselves becoming roommates with a talking bird. He further uses this as a metaphor for his creative process with AI music tools. The video then transitions to a live performance from his AI-generated band, Chiptune, and ends with a thought-provoking song titled "Change Your Life".

The content is underpinned by a deeper exploration of the endless and evolving system of communicating ideas about sound and music, and how new technological tools have historically influenced and expanded the scope of music. Furthermore, a walkthrough of how AI tools were used to generate musical and visual content is included, emphasizing the limitless possibilities of these technologies in the field of music.

If you're interested in music, technology, and the intersection of the two, this is a must-watch video. Get ready to have your mind blown, your horizons expanded, and potentially, your life changed.

Video Transcription

• 00:00:12 Hello.

• 00:00:13 Hello.

• 00:00:14 Hello.

• 00:00:15 Hello.

• 00:00:16 Hello.

• 00:00:17 Hello.

• 00:00:18 Hello everyone.

• 00:00:19 My name is Dan Deacon.

• 00:00:20 Thanks for coming here early.

• 00:00:22 It's a real pleasure to be sharing the stage with some of my all time favorites, Gmail

• 00:00:27 and Google Calendar.

• 00:00:30 I'm going to play some songs and a lot of the content is going to be made using Fnacke

• 00:00:38 and Bard and MusicLM I'm just going to get started.

• 00:00:44 This first one is a song of mine called When I Was Done Dying.

• 00:00:46 The video was generated using the lyrics to this song.

• 00:00:49 Thanks so much.

• 00:00:51 Check, check, check, check, check.

• 00:00:59 When I was done dying.

• 00:01:00 When I was done dying.

• 00:01:11 When I was done dying.

• 00:01:34 When I was done dying.

• 00:01:41 When I was done dying.

• 00:01:53 When I was done dying.

• 00:02:05 When I was done dying.

• 00:02:08 When I was done dying.

• 00:02:10 When I was done dying.

• 00:02:13 When I was done dying.

• 00:02:16 When I was done dying.

• 00:02:19 When I was done dying.

• 00:02:22 When I was done dying.

• 00:02:25 When I was done dying.

• 00:02:27 When I was done dying.

• 00:02:30 When I was done dying there at night in the dark in unseated rooms, so I thought maybe

• 00:02:38 we could all just try to do a guided visualization so we can all get on the same place.

• 00:02:44 I'm sure a lot of you traveled here today.

• 00:02:46 So let's just fully relax our shoulders, feel our bodies in the seats, our feet on the floor.

• 00:02:54 I used BARD and I was like, how can I structure a guided visualization for this event?

• 00:03:01 They gave me some suggestions.

• 00:03:05 So let's envision now that we're slowly, slowly lifting up from our seats, floating into the

• 00:03:12 air, and let's imagine the air is not really cold.

• 00:03:16 Let's imagine the air is a wonderful, wonderful temperature.

• 00:03:20 We're floating here above the Shoreline Amphitheater.

• 00:03:23 We see everyone else below.

• 00:03:25 They're floating up with us.

• 00:03:27 We're becoming mist, vapor, and there we see a beautiful, beautiful entity come towards

• 00:03:34 us.

• 00:03:35 And what is it?

• 00:03:36 It's a bird, a bird with lips.

• 00:03:41 Don't pay too much attention to the lips, but the bird has lips.

• 00:03:44 And the bird says to you, hey, I just got a two-bedroom apartment.

• 00:03:49 I'm looking for a roommate.

• 00:03:50 And you think, wow, I never thought about relocating, but what neighborhood is it in?

• 00:03:55 And the bird says, it's in this neighborhood that you've always wanted to live in.

• 00:03:59 You say, that sounds great.

• 00:04:00 Do you live with anybody else?

• 00:04:02 You're like, no, but I have a lot of friends over.

• 00:04:03 We played board games.

• 00:04:04 Do you like board games?

• 00:04:05 And you're like, yes, I do love playing board games.

• 00:04:07 I have friends over.

• 00:04:09 We play board games.

• 00:04:10 They're like, great.

• 00:04:11 Maybe my friends can become your friends and we'll start a whole community.

• 00:04:13 And the bird's like, great.

• 00:04:14 The only thing is it's kind of a small kitchen.

• 00:04:16 So I don't like that many kitchen appliances.

• 00:04:18 And you're like, that's wonderful.

• 00:04:19 I normally eat out, but I'm trying to cook more at home.

• 00:04:22 They're like, that's wonderful.

• 00:04:23 We can make meals together.

• 00:04:25 So now that we're all on the same page, we're re-centered here.

• 00:04:31 Let's just think of how beautiful the world would be if there was a bird with lips that

• 00:04:35 invited us to live with them and we could make wonderful, wonderful meals together.

• 00:04:38 And that's sort of where this whole process has felt like.

• 00:04:40 It's felt like being a bird with lips and finding a new roommate and making work with

• 00:04:45 them.

• 00:04:46 So if that makes any sense, I'll explain more.

• 00:04:50 As a composer, it's always interested me in that composing music is an endless and evolving

• 00:04:55 system of finding ways of communicating ideas about how to make sounds.

• 00:04:59 Music has always been expanding with ways to share ideas and about how to make sounds

• 00:05:04 while exploring what sounds are available to us that can be called music.

• 00:05:09 Another thing that fascinates me as a composer is that throughout music's history, advancements

• 00:05:13 in technology have always led to and coincided with advancements in musical thinking.

• 00:05:18 New tools lead to new instruments, which lead to new musics, which leads to new boundaries

• 00:05:22 to break, which leads to a restart for new tools.

• 00:05:25 At one point in time, the trombone was a reflection of cutting edge musical technology.

• 00:05:30 And like many technologies, when the first trombones entered the scene, they were viewed

• 00:05:34 as highly controversial pieces of music tech.

• 00:05:37 The same could be said for player pianos, synthesizers, synchronizing sound with film,

• 00:05:42 home recording studios, and MIDI.

• 00:05:43 I love all these things.

• 00:05:45 And when I started getting into making computer music as a teenage nerd on my family computer,

• 00:05:50 all of those tools were readily established and antiquated, but they were there for people

• 00:05:54 to make music with.

• 00:05:56 And ever since those early days of me playing with toy keyboards and primitive MIDI software,

• 00:06:00 I've been trying to explore new ways of making new sounds.

• 00:06:04 So when Google invited me to come meet with some of their AI researchers and talk about

• 00:06:08 what they've been working on, I was pretty ecstatic to get the chance to be one of the

• 00:06:11 first people to use these tools and see how they're being shaped and to meet with them

• 00:06:15 and get a chance to make new music with them.

• 00:06:18 Last month, I was pretty lucky to spend a few days with the researchers at MusicLM,

• 00:06:22 Finacki, and Bard.

• 00:06:23 They showed me the tools that they've been making and explained the concepts behind how

• 00:06:27 they worked and allowed me to experiment with making sounds, music, and visuals with them.

• 00:06:31 It felt a lot like those early days of sitting in front of a computer and having no idea

• 00:06:36 what I'm doing or taking out a Casio keyboard and hitting buttons and see what happens.

• 00:06:40 I had no idea what happened, and that was sort of the fun part, just mashing it and

• 00:06:44 seeing what it is and making something new with it.

• 00:06:46 It reminded me a lot of being a child again.

• 00:06:49 So you'd hear that drum beat, and that would start, and the autocords would start, and

• 00:06:55 the outputs from AI felt the same way.

• 00:06:56 They would spark new ideas, they created building blocks to make new songs with.

• 00:07:01 So the music that accompanied our guided visualization about becoming roommates was Duck with Lips.

• 00:07:05 That was generated using the text prompt in text to music using sad, moody, new age music,

• 00:07:11 simple piano, and synthesizer.

• 00:07:14 Once we found an output that we liked, we started experimenting with it and processing

• 00:07:17 it with Sing Song to create new variations to accompany the text to music outputs.

• 00:07:22 We fed those results into themselves countless times, making dozens of iterations to explore,

• 00:07:27 and those results were the song you heard again while I was rambling on about the bird

• 00:07:29 with lips.

• 00:07:31 The text to music model output is available as part of the AI Test Kitchen.

• 00:07:34 You can sign up for the waitlist now.

• 00:07:37 We also made a new track that we're about to play using a single word prompt, Chiptune.

• 00:07:43 The name of the band is Chiptune, and our album is Chiptune, and if you'd like to also

• 00:07:48 join the band, you're welcome to join the band.

• 00:07:49 This is our first show.

• 00:07:50 We're all members of the band, so thanks for joining the band.

• 00:07:52 You can also quit the band at any time, and let's see how it goes.

• 00:08:02 So we put the word Chiptune in, and this is the output that we got.

• 00:08:09 We realized we needed another section of the song, so we fed it back into itself.

• 00:08:16 And that gave us this B section.

• 00:08:21 We knew it needed to be a full song.

• 00:08:23 It couldn't just be these two riffs, but as you know, every song needs a chorus, so we

• 00:08:27 made the opening part our refrain.

• 00:08:29 And here we are back at it.

• 00:08:31 We're like, all right, well, it's a Chiptune song, so maybe we should add some sort of

• 00:08:37 adventure theme.

• 00:08:38 So we're like, all right, well, now we know what's got to come up after this.

• 00:08:43 So what came after this?

• 00:08:45 It was the breakdown, because every song needs a breakdown.

• 00:08:49 We're like, all right, we're going to be at the shoreline.

• 00:08:56 People are going to be going nuts in their seats.

• 00:08:57 There's going to be mosh pits everywhere, a crowd surfer, so when the breakdown's done,

• 00:08:59 it's just going to be, we need this part, the full-on release.

• 00:09:06 So this part came as kind of an unexpected little gift.

• 00:09:22 We didn't really know that we'd be getting this.

• 00:09:25 And that was the fun part of this discovery, being like, what we would put in would be

• 00:09:31 like, finding a new sound that we didn't even expect to create.

• 00:09:33 But then of course, we go back to our beautiful refrain, which we fell in love with at the

• 00:09:46 beginning.

• 00:09:48 Thank you very much.

• 00:09:58 That was our first show for our band Chiptune.

• 00:10:05 Some of the other band members are here in the audience, and some of our band members

• 00:10:10 don't exist in the physical realm.

• 00:10:12 I've never been in a band where the band members aren't physical entities.

• 00:10:15 So all right, now that we've been basking in the Chiptune, let's return to our guided

• 00:10:22 meditation.

• 00:10:23 We've got a long day of Google ahead of us, and now let's place ourselves in a beautiful

• 00:10:30 woods where our roommate's house is.

• 00:10:34 We're wandering through the woods.

• 00:10:36 We're coming back from a long day, and we're thinking about all of the Google jokes that

• 00:10:42 we could make.

• 00:10:43 And you're excited to tell your roommate the joke that you came up with.

• 00:10:47 And you're like, and then they appear.

• 00:10:49 You're like, hey, duck with lips.

• 00:10:50 Doing this event for Google, and I'm thinking about using this joke.

• 00:10:53 The duck's like, what is it?

• 00:10:56 You just get mesmerized by the feathers.

• 00:10:59 You're pulled deeper and deeper into this ethereal realm of pure feathers.

• 00:11:03 The duck says, hey, I'm sorry, but I finished the oat milk.

• 00:11:06 And you're like, oh, I was really looking forward to having that with my cereal.

• 00:11:10 Like, that's okay.

• 00:11:11 I'll go out and get more.

• 00:11:12 And you're like, no, that's fine.

• 00:11:13 I have this joke I want to tell you.

• 00:11:15 And they're like, it's this more serious matter.

• 00:11:16 It's more that I know you've been making oat milk at home, and we talk specifically about

• 00:11:21 not having so many kitchen appliances.

• 00:11:22 It's a small kitchen.

• 00:11:24 But the contractor we wanted has the waiting list open, and I think we can expand the kitchen.

• 00:11:29 And your heart just fills with pure love of the idea of blowing out the kitchen, adding

• 00:11:34 more walls, adding more cabinetry.

• 00:11:36 So you can make almond milk at home, oat milk at home, maybe even build a little stable

• 00:11:42 so you can have other sort of mammalian milks.

• 00:11:44 And the bird is like, this is the best relationship I've ever had.

• 00:11:47 And you're like, this is wonderful bird with lips.

• 00:11:49 Thank you so much.

• 00:11:51 And then you go back to the house, and you're trying so deeply to remember the joke that

• 00:11:55 you were going to make.

• 00:11:57 And then you remember in the streets, it's Google Maps, and in the sheets, it's Google

• 00:12:02 Sheets.

• 00:12:03 And the duck's like, I think if you delivered it earlier, it might have landed a little

• 00:12:07 better.

• 00:12:08 Why don't we just go relax back in our home, drift away into the ether, and then tomorrow,

• 00:12:12 we'll find a new way to find that joke into our everyday life.

• 00:12:15 And you just wish so badly that the duck with lips was real.

• 00:12:20 So if we could all now just collectively, with all of our ability and all of our might,

• 00:12:25 try to manifest a duck with lips.

• 00:12:28 And if we think hard enough, perhaps we can put this creature into reality, and we can

• 00:12:33 share this utopian life with this beautiful roommate, the duck with lips.

• 00:12:38 So this will be my last song.

• 00:12:39 And then I'm going to do a five-hour set later, so don't worry.

• 00:12:44 This song is called Change Your Life.

• 00:12:46 And it seemed like a good thing to do right before this talk, where I think all of our

• 00:12:50 lives are going to drastically change.

• 00:12:52 And thanks so much.

• 00:12:54 Again, my name is Dan Deacon.

• 00:12:57 And again, these visuals were generated using Pinaki.

• 00:13:00 Tonight, tonight, tonight, you're gonna change your life.

• 00:13:06 You're gonna change your life.

• 00:13:09 Tonight, tonight, you're gonna change your life.

• 00:13:12 Tonight, tonight, you're gonna change your life.

• 00:13:14 Tonight, tonight, you're gonna change your life.

• 00:13:16 Tonight, tonight, you're gonna change your life.

• 00:13:18 Your life, your life, your life, your life, your life, your life.

• 00:13:22 Tonight, tonight, tonight, tonight, you're gonna change your life.

• 00:13:25 You're gonna change your life.

• 00:13:27 Yes, you can.

• 00:13:28 You know you can.

• 00:13:29 Yes, you can.

• 00:13:30 You know you can.

• 00:13:31 Yes, you can.

• 00:13:32 Yes, you can.