Poets & Thinkers
Poets & Thinkers explores the humanistic future of business leadership through deep, unscripted conversations with visionary minds – from best-selling authors and inspiring artists to leading academic experts and seasoned executives.
Hosted by tech executive, advisor, and Princeton entrepreneurship & design fellow Ben Lehnert, this podcast challenges conventional MBA wisdom, blending creative leadership, liberal arts, and innovation to reimagine what it means to lead in the AI era.
If you believe leadership is both an art and a responsibility, this is your space to listen, reflect, and evolve.
Poets & Thinkers
Multisensory Beings: How neuroaesthetics shapes the future human-machine interaction and art – with Matthew Bennett
Humans are multisensory beings. What if the tiny sounds you hear from your devices every day are literally vibrating through your body, changing your nervous system, and collectively creating decades of audio pollution? And what do we do about it in an age where generative AI is likely going to add even more noise?
In this fascinating episode of Poets & Thinkers, we explore the profound intersection of sound, technology, and human experience with Matthew Bennett, a composer, sound artist, and sensory designer who led sound design at Microsoft for 12 years. From his home studio in Seattle, Matthew reveals how he shaped the sonic experience of billions of people worldwide while pioneering a new paradigm for technology sound design.
Matthew takes us on a journey through the science of sound as sensory experience – not just something we hear, but a form of touch that vibrates our entire body and changes our physiology. He shares mind-blowing insights about how Microsoft’s tiny notification sounds, when multiplied across hundreds of millions of users, created decades of sound pollution daily – and how his team cut 10 years off that global audio footprint by shortening sounds by just one second. Through the lens of neuroaesthetics and multisensory design, Matthew illustrates why our digital experiences are always multisensory whether we intend them to be or not.
Throughout our conversation, Matthew challenges the current AI music generation hype, revealing how these tools expose the formulaic nature of popular music while lacking the human intention and authenticity that gives art its soul. He advocates for a “do no harm” approach to sound design, emphasizing the importance of designing silence and understanding that unexpected sounds can hijack our brains and trigger fight-or-flight responses. His vision for Musical Sensory Environments and precision therapies offers a glimpse into how sound can heal rather than harm.
In this discussion, we explore:
- Why sound is actually a special form of touch that vibrates through your entire body
- How tiny notification sounds create decades of global audio pollution daily
- The ethics of multisensory design and the responsibility that comes with scale
- Why AI-generated music reveals the formulaic nature of popular genres
- How neuroesthetics can become essential literacy for designers and leaders
- The difference between human intention and statistical pattern matching in creativity
This episode is an invitation to understand sound as a powerful force that shapes our digital ecosystems, our physical well-being, and our human connections – and to approach the creation of sensory experiences with the care and intention they deserve.
Resources Mentioned
- Jaron Lanier’ work
- World Health Organization (WHO) research on noise pollution as global health crisis
- Neuroaesthetics research and fMRI studies on brain responses to sound
- Musical Sensory Environments – Matthew’s pioneering approach to immersive audio
Connect with Matthew Bennett:
Website: https://soundandsensory.com/
Get in touch: ben@poetsandthinkers.co
Follow us on Instagram: https://www.instagram.com/poetsandthinkerspodcast/
Subscribe to Poets & Thinkers on Apple Podcasts: https://podcasts.apple.com/us/podcast/poets-thinkers/id1799627484
Subscribe to Poets & Thinkers on Spotify: https://open.spotify.com/show/4N4jnNEJraemvlHIyUZdww?si=2195345fa6d249fd
Welcome to Poets and Thinkers, the podcast where we explore the future of humanistic business leadership. I'm your host, ben, and today I'm speaking with Matthew Bennett. Matthew is a composer, sound artist and sensory designer pioneering the development of musical sensory environments for immersive listening experiences. As a former chief sound designer at Microsoft, his sounds and compositions have been heard by billions of people worldwide, which has also earned him the title of the most listened-to composer no one has ever heard of. With a background in art, technology, performance and musicology, matt seamlessly fuses different traditions, disciplines and music cultures to create a unique and highly personal musical and sensory language.
Speaker 1:Ever since we crossed paths working together at Microsoft, I've been a huge fan of Matthew and his work. Over the years, we've had many conversations about the power of sound as touch, how to visualize complex data through sound and the profound impact of the latest neuroaesthetic research. Now that AI provides the creation of sounds and compositions with just a text prompt and a push of a button, as well as enables true multimodal experiences, it's critical that we dive deeper into both positive applications and the negative consequences of mass-produced audio experiences. Throughout our conversation, matt challenges the current AI music generation hype, revealing how these tools expose the formulaic nature of popular music, while lacking the human intention and authenticity that gives art its soul. He advocates for a do-no-harm approach to sound and sensory design, emphasizing the importance of designing silence and understanding that unexpected sounds can hijack our brains. His vision for musical sensory environments and precision therapies offers a glimpse into how sound can heal rather than harm, so let's dive in.
Speaker 1:If you like the show, make sure you like, subscribe and share this podcast. Hi Matt, where does this podcast find you?
Speaker 2:Hey Ben, I am currently at my home studio in Seattle, Washington.
Speaker 1:Great. So why don't we get started and you tell us a little bit about yourself? For those of you who are listening who don't know Matt yet, maybe, matt, you give us a little bit of an insight into who you are, what you've done so far, what drives you, and then we'll jump into all the questions I have for you.
Speaker 2:Yeah, sure. So I'm a composer, a sound artist and a technology sound designer. I led sound design at Microsoft for about 12 years, where I was responsible for the sound of all the major platforms. While I was there, I also founded and directed the Sound and Sensory Design Program and created a new paradigm for technology sound design, which I think has helped change the way that sound is designed for modern digital experiences.
Speaker 1:And you're almost certainly underselling the impact that you've had with the work that you've done specifically at Microsoft, and I think everyone who has in any form touched a Microsoft product hardware or software has heard your work, even though they might have not even noticed it, which I think is incredible.
Speaker 1:And there is actually some YouTube videos out there from the past where you, I think, are doing an incredible job explaining all the thinking, the craft and really also the science, the research that went into all the work that you and the teams that you directed were doing.
Speaker 1:So we'll put those in the show notes for people to enjoy, because I think it's incredibly inspiring and something that, as I said, most designers even probably not consciously think about Just making sure that everyone understands the gravity of the work that you've done and the scale, which is one of the many reasons I wanted to talk to you today.
Speaker 1:So maybe just jumping off from there, it's actually a good transition into one of the many reasons I wanted to talk to you today. So maybe just jumping off from there, it's actually a good transition into one of my first questions, because one of the things that I always appreciated about the way you framed your work was actually not just about sound. It was actually more about sensory design, and I remember one of the conversations that you and I had a while back where it blew my mind when you shared the fact that sound isn't just something that we hear, but also something that we experience through other senses, specifically through touch. Why don't we talk a little bit more about sensory design? What is it, how do you think about it, how do you design it? And I would love to you know, expand a little bit more on that, because I think it's so profound.
Speaker 2:Yeah, thanks for the context. First of all, as you know, when you're working on a global platform, everything you do is amplified and heard by hundreds of millions of people around the world, which is a great blessing and there's a lot of potential there, and it's also can be kind of a curse. I mean, there's a huge responsibility in that. So I don't take too much personal credit for for the global reach of of my work, although I'm really grateful for it. And you know there are a lot of ethical, logistical, but also ethical and cross-cultural implications of designing sounds that that are going to become part of people's lives, part of private and public and family soundscapes. You know how do you do that so the sound actually sits in the context of people's lives in a way that is that is welcome and not annoying and in a way that that makes their digital experiences richer and and and gives them more depth.
Speaker 2:So someone had called me the most, the most hurt composer you've never heard of, and I'm totally fine with that when we ran the numbers one time just for, like it's these little tiny sounds, right, they're like micro compositions, so they're a second.
Speaker 2:Two seconds is a long one. You know, half a second is pretty typical, and we were. We just kind of we're trying to quantify so that we could make a more healthy digital soundscape for everybody, digital soundscape for everybody. So one way we did that was just by measuring the amount of sound that we're putting in the world every day with, say, a Windows operating system or Teams or Xbox UI or something like that, but like just to keep it to Windows for a minute. You know, it turned out to be those little tiny sounds multiplied by hundreds of millions of users, multiplied by multiple times a day, hearing multiple times a day, and even assuming that, say, maybe half the people have their sounds off at any given time, that still is like I can't remember the exact number, but it was like years, it was like decades of sound into the world every day. And you know that really changes our perspective about how we think about that. So you, so we eventually did some things to shorten the sounds and get them down.
Speaker 2:You know, like we cut 10 years off the amount of noise pollution every day, and that's something we definitely don't want to do is noise pollution. You know we think of it, as I should say I used to think of it and I still do in my personal work away from Microsoft as a digital audio ecosystem.
Speaker 2:If you think of, like a rainforest, layers of sounds, right that all kind of work together ideally and can be very harmonious and create this, even a serene experience.
Speaker 2:Meanwhile you're getting, just you know, a huge amount of information from all directions around you. You know that's sort of the ideal is the tropical rainforest right, you've got the canopy, you've got insects, you've got anyway, so all these different layers. So we sort of think of it like that and, yeah, just so, in our effort to make it healthy, we actually just by shortening one sound by one second, we're actually globally kind of doing this, this mitzvah, this good thing, to make the world a little better place and make our sounds a little more meaningful and useful and and kind of cut down on people's overstimulation. Because is really one of the main challenges in customer experience these days, if you want to connect the design to the business, is people are overwhelmed and have these fragmentary experiences and there's this customer kind of malaise around the world. And I'm not speaking about Microsoft customers specifically, but just people in general in the digital world right Whether I'm working for a major client or a small client.
Speaker 1:So one of the things that I want to kind of draw out here is really those considerations that you just laid out that, I think, give such a great perspective of how to design products at scale right, because it's one of those terms that is thrown out very easily in terms of, well, we want to scale this product or we're designing at scale. Well, I think those numbers and the way you illustrated those really communicate so well is the real life impact that the work has on the life of hundreds of millions, billions of people and that, multiplied over time, is incredible. Not even to speak about the, as you said, the cultural implications, and there's probably multiple factors here. One is making sure that whatever you create and compose actually works in the cultural context really around the world, which in itself is probably a huge challenge. I would love to hear some more about that.
Speaker 1:But also how, because you're working at scale and because your products are used by billions of people every day, your sound and your composition becomes part of the cultural heritage, if you want it's part. You know, we all know certain sounds because we've heard them so many times and it becomes just part of the culture. So what are your thoughts on that and how I'm speaking, especially from a perspective of we're at this very transformational point in time where even more stuff is being created, and part of why I want to have this conversation with you is to really learn and and hopefully communicate to to others who are in the situation now, to really discern what should be composed, what should be created, what should be put out in the world, and how do we consider the impact of the work that we're putting out, especially because the scale and the speed of creation is increasing so drastically.
Speaker 2:Yeah, yeah, wow, a lot to think about there. Let me let me pull one thread out and we'll see if that can lead to Great To two others. The most fundamental thing for me in approaching sound design, whether it's for a global platform or for one person, is to think about sound as sensory experience. And this kind of gets back to your earlier question too. So, you know, sound is vibration, it's physical energy and, as you mentioned earlier, we consider it a special form of touch, and in a scientific, technical sense it actually is touch. You know, it's actually going inside your head, it's vibrating your whole body, it's haptic energy. So sound has a certain kind of intimacy in it. Even if it's a very public thing, right, it's also very personal. It transverses the thresholds of our perception and the boundaries of our body in space, right, the sound actually vibrates into our body before it's then converted into electrical and chemical energy information that gives us the mental representation of what we are experiencing. So that's an incredibly powerful and kind of humbling thing to realize about sound.
Speaker 2:So it's physical vibration, but it's also emotional energy, right, physical energy, but also like psychological and emotional energy. And we perceive all this vibrating energy with our whole body, not just, not just our ears. And you know, a fun thing to do for someone to like experience that in a personal way is like next time you're in the shower, just plug up your ears and see what you hear. It will blow your mind of the frequency spectrum and and the rhythm and how much you're actually hearing through your bone conduction, in this case, like of your skull.
Speaker 2:It sounds sounds kind of weird, but it's true and it's. It's visceral and everybody can feel that you know. And then to think that that's always happening, that's one reason our voices sound different to ourselves than when we hear them on a recording to other people. We're hearing through the air, but we're also hearing through our body. We're hearing our voice literally vibrating through our own body, right? So that's why we have a lot, why we usually like the way it sounds to us better than it sounds to other people, until we get used to that.
Speaker 2:So, yeah, that vibrating energy that is essentially like touch or giving someone a big hug or a little tap on the shoulder sometimes, depending on the quality of the sound, actually changes the nervous system.
Speaker 2:So to translate that to a global platform, you know you're creating all these multi-sensory experiences for people around the world and I just want to say that you know life is multi-sensory. We're always creating multi-sensory experiences. Life is not a flat screen. People are hungry for richer experiences. So even if they are only working on a flat screen and only having a visual experience, real life is multisensory and every digital product, brand interaction that we design is a multisensory experience, whether we intend it to be or not. And if we ignore that, we're ignoring a large amount of human sensory potential, which is not good for people. It's not good for people's mental health, it's not good for our design, it's not good for business. You know, frankly, I'm surprised that more businesses are not embracing multisensory design as a response to the commodification, standardization and other things that are going on in the design world and automation and AI. We can talk about that. That's a huge topic.
Speaker 1:Yeah, yeah, and I want to get to. I definitely want to get to that for sure, because there's a lot of questions I have there for you too. But this last point is actually really interesting. It's one of the questions I wrote down was you know your perspective and also any of the research that you know that has informed your work on? We are multisensory beings. Throughout our our day, throughout our life, we are experiencing the world um through multiple senses, and those are, you know, different for for every person, but, generally speaking, most of our experiences are multisensory. Now is there what's the evidence and what's the research in terms of how companies can actually build more meaningful and in this case, actually I'm thinking mostly more engaging experiences, products that, in turn, build a stronger connection between the human that's using the product and, of course, then the brand, the company, whatever you want to call it. So what's your take on that?
Speaker 2:Yeah, there's actually a lot of work that's been done and I think there's still more to do in that area.
Speaker 2:But actually, taking some of the scientific literature, some of it looks specifically at digital experiences or product experiences, whether it's the food industry, restaurants, people eating, how sound and music modulates the experience of taste and smell and other things like that. Fascinating um, uh, modulates the experience of taste, um and smell and other things like that. Um, so there's a lot going on in some of its kind of business and and sector focused, and some is more um, uh, a little more general, uh, at uh, um, like scientific research level and it, it all can be very, very useful and I find a lot of the literature to be very inspiring. In fact, I think, um, you know, in terms of you and I have talked about neuro aesthetics a little bit in the past and I think, increasingly, the research is so useful for actual practical real world design challenges, whether it's trying to have make a business or brands connect more, you know, for more deeper engagement, or or how to orchestrate a product experience, whether it's in AR, vr or just you know traditional flat screens.
Speaker 2:Multisensory thinking and neuroaesthetics can really inform that it should be a part of every designer's toolkit, I think.
Speaker 1:Yeah.
Speaker 2:I mean, and the easiest way is obviously visual designers can be more aware of sound and start thinking about that. You know, have designated sound ambassadors on the team. Even if you're not a professional sound designer, you know a lot more about it than you think if you're a visual designer. In touch with those experiences and some very simple things and toolkits that we can design, and some things that I have designed for certain groups, can add a lot to a brand experience or a product experience or a digital experience.
Speaker 1:Yeah, yeah, go. That's really insightful and I want to just ask kind of a pointed question here, because I agree with you in terms of, you know, neuroaesthetics becoming part of the. The more we learn about the brain and the way we, you know, impact the brain and the human physiology through the work that we're doing as designers and product makers and business leaders, that needs to become part of the toolkit.
Speaker 2:Yeah, yeah, totally agree.
Speaker 1:The question I have is where is the line, and how do we measure the line between doing good and doing harm? Right, because there is this, this very fine line where, once we cross over, we're getting into the territory of manipulation. And because we can I actually had this conversation just a week ago with some top design executives at big, big firms where we can alter people's physiology we have for a long time, but it's one of those things where now, more and more, the research is actually catching up and we have the insight. So what's your take on? I mean, there's a real ethical question here that we need to spend more time kind of wrestling with and also developing, as part of the toolkit, I guess, ways to assess where we're doing harm and where we're doing good.
Speaker 2:Yeah, yeah, it could.
Speaker 2:Understanding multisensory design and and and the fact that you know, people's senses are not separate, they're continually merging and overlapping, and and everyone has this innate capacity for some level of of synesthetic experience.
Speaker 2:I mean, we really all do, even if we don't have like full-blown synesthesia. It's like this baked in superpower that we all have a little bit and that designers can harness and anybody communicating can harness for good effects or or for, you know, not so good intentions, and and in the same way that all types of design can be used to, to digitally in the digital world, let's just say, to improve experiences and quality of life and mental health, they can also be used to to manipulate people and and there can be dark patterns that are, that are are multi-sensory well as as anything else, and I think we have to guard against that. And and you know, for sound, we kind of have a, at least in my personal practice and the people that I've worked with and mentored you know, I have kind of an ethical, just a real world ethical approach that I use and we always start with, like you know, first do no harm.
Speaker 2:We're not doctors, but we realize that that sound can have a lot of power at one just real common experience of that that I can point to, that people can probably relate to in their own lives, is if you hear an unexpected loud sound, you can think about that. What that feels like, right, it immediately hijacks your brain and before you can even consciously think about it, you're within like just a few milliseconds. Your body has started to respond because of the anxiety and the fear and the alertness that you have to have for danger in an evolutionary sense right.
Speaker 2:So we still carry that in a very active way.
Speaker 2:It probably happens multiple times a day to us and the thing about that for digital experiences and for the kind of design work that I do is understanding that it doesn't have to be a loud sound to be stressful and produce anxiety or to scare the hell out of you. It just has to be something that's unexpected. It can be a very quiet sound, but if you're in a very quiet environment, the wrong sound at the wrong time can really mess up your flow. I mean, not only does it interrupt your, your attention, that you then have to kind of struggle to get back, often at an unconscious level. People aren't always aware of that, but this happens. Sound can either facilitate the rhythm of attention and and your kind of task orientation, or giving you a moment to sit back and and process and integrate your experience and reflect a little bit. It can push you forward or it can kind of keep you in the present or actually even pull you back and out of the experience and in a good way or in a bad way.
Speaker 2:So kind of got layers going on in this conversation. But there's the ethical idea, there's the neuro aesthetics of of sound and there's the idea that it's just, it's. You know, once we start to put senses together I mean any, any sensory experience can be powerful and moving. Once you start to put them together. The good news is that they become more powerful, right. The more channels that we're perceiving a certain experience with, the more sensory channels, the better we're remembering it. The more immersed we are in it, the more immersive it is. That's good news. That's also bad news if it's a negative experience, right, it just, it's literally.
Speaker 2:You know, if you get one of those loud, unexpected sounds, your cortisol spikes when you look at those kinds of experiences on again on a global level, beyond just like a global tech platform. But noise pollution is one of the single most challenging and problematic health situations in the world right now. I think the World Health Organization has it as number one or number two in terms of world health problems because of all the cascading effects you get from that. Like, statistically speaking, more people die every year because of noise pollution than need to. Wow.
Speaker 1:Yeah.
Speaker 2:And you know, I don't know how they arrive at that it's very complex and it's not just saying like sound killed this guy or anything like that, but just at a population level, looking at it from a public health perspective, noise pollution right now is kind of where it's almost kind of where smoking used to be like 10 or 15 or 20 years ago. People are just starting to realize how much it impacts their experience in a negative way, whether it's environmental noise or digital noise from our products. So one thing that I've always tried to do, and with my teams, is to be sure that we're aware of that Right and that we're not triggering that fight or flight response. We really go way out of our way to make things calm and to design silence Right. I mean, the silence is really important.
Speaker 2:Not only does it make the sounds more meaningful when you finally do hear them, cognitively you we attend to them differently, because when they're there all the time, you start to tune them out, but it lets your, gives your body and your brain a chance to relax and integrate and be more present for for other experience. So so it's about the meaning of the sound too, and I think that's one of the main ways that we can um and haptics is, is is another extension of that. So the sound and haptics I think very strategic, careful, I would almost say loving placement is is important and also knowing where to leave sensory silence for people is is really important. That's actually one of the most important things we can do.
Speaker 1:Is is is not place the sound um everywhere right which is a great segue into my question that I have around generative AI, and it's great to hear you talk about in depth, about the approach and the considerations that you have built and actually, to a certain extent, I would say even shaped some of the paradigms and frameworks for at least the digital industry, if not beyond that, because of how deep your impact was reaching, but also pretty much the scale of Microsoft as a company, of course, in the context of generative AI.
Speaker 1:I have a few questions for you, and they all sort of go hand in hand. One just the fact that now and I've played with some of the new platforms that are out there, where you can basically generate anything from songs to sounds with a few prompts, right so it is both exhilarating and inspiring, but it's also almost terrifying how quickly you can put stuff out into the world, and it has a lot of kind of comparisons to what we see in in fashion, what we see in, you know, fast food, and so I wonder how you think about the fact that now, on the one hand, there is or is this whole new tool stack that makes the creation of sound available to everyone. There's an artistic bucket of questions that I have, which we'll get to probably in a little bit, but just maybe a general take, first and foremost on all of these new generative AI tools, and then we'll dive deeper into more the kind of creative process and all of that.
Speaker 2:What's going on with generative AI right now. I'll just maybe keep my comments focused on music or art in general. It's a huge topic. There's lots of different kinds of AI, you know, and I think that gets lost sometimes, and I almost don't like to call it AI.
Speaker 2:Jaron Lanier, who you might know, says you know that actually gives it too much agency and frames the narrative in a way that is not necessarily healthy. I prefer to think of it as very sophisticated machine learning. You know, it's math, it's statistics, not that it can't point to some really wonderful, interesting, complex, emergent sort of things that happen. But let's be real. I mean, I think there's a lot of hype, right, and I'm not the best person to offer a critique of that, but I think we're all experiencing that. So, all right, let's be real. Some of those music tools I won't name any individually are pretty interesting, right. And to think that in some ways it makes me realize when you hear someone just generate a song like that with a few text prompts, or when you do it yourself, I recommend that you do it. It's an interesting experience and it's a learning experience, right. For me it really highlighted how formulaic a lot of the stuff that we're used to hearing is. If you can make a convincing pop song with a few pop, a few text prompts, you know there's a pretty statistical consistency there about what makes that genre and that style sound, the way it does that language, the syntax, which I think is really interesting. You know it's not like AI invented that.
Speaker 2:People have been studying that for a long time and that's one way we know what a style is. It has certain patterns. If it didn't, things would be a lot different and sometimes and part of the excitement and anticipation and, I think, surprise in musical experience is messing with those expectations. Sometimes you know lots of interesting stuff going on with combining styles, right, that used to be a real big deal. If you combine jazz and rock or jazz and classical, now that's not a big deal at all. Everybody's doing it in really, really interesting ways.
Speaker 2:I don't know if AI is doing that because it can't do it until people have done it and it's been trained to do it. It cannot take just for example, or actually it would be interesting to explore. I don't want to say that it can't because I'm not an expert, but now it makes me think it would be interesting to prompt an AI music creation tool to combine elements of, say, jazz and Indian classical music or Western classical music from the 18th century and Indian classical, raga or something, or gamelan, indonesian gamelan Just really disparate styles, to see what it understands and if it can find the places of syncretism and synergy between those styles that can allow it to create in a more or less original language or a new language, kind of based on those things. That's the kind of thing a person would do.
Speaker 2:I'm giving away my. I'm not giving away. I'm showing you how I think about my own creative process. Mm-hmm, Somebody had to invent that style and in most cases, many, many different people had to work together to collectively invent these styles and these musical cultures and traditions over decades, if not more, depending on what style you're talking about, before AI can be trained on it to find all the patterns and then be able to reassemble them.
Speaker 2:So, AI itself can't as.
Speaker 2:I understand it cannot actually do the actual creative work. Now, creative work is a big topic, right, and that's a huge umbrella. That can mean a lot of things, from modifying someone else's idea to having whole new kind of categories of combining disciplines or styles or ideas from totally different worlds. But let's just keep it simple. If you tell AI to write a Bach fugue and computers have been able to do that now for many decades, even before there was like AI and sophisticated machine learning it's a very rule-bound system. Once it knows the rules it can do that, but it cannot create the rules. Now, maybe there is kind of reflexive things that they can do now to help AI create new rules, but that's a whole nother level of creative thinking. If the machine didn't know what a box fugue was and didn't know the syntax and the grammar, it wouldn't be able to create a fugue.
Speaker 2:Someone had to create that, and that's where I think it really highlights the importance of human creativity. It can help us see what we're doing. That's where I think it really highlights the importance of human creativity. It helps, it can help us see what's what we're doing. That's kind of formulaic, I guess, which we all do because there has to be consistency and coherence and formulas are important, but it also helps us see, I think, how we can be more creative, whether we're using that as a tool for a springboard or just thinking about it conceptually. Board or just thinking about it conceptually, like you know, if this machine can write, if this program can write a pop song as good as I could given a few days, or, in my case, better than I could, because I don't write pop songs very well um, you know what? What does that say about a pop music and about and about the human element in there? That's right.
Speaker 1:so so, yeah, no, it really exposes a really interesting dynamic here, or fact, potentially, where, yeah, you know three chords, three minutes long, pretty much the same stories and progression, and all of that how, how, you know, creative. Is that really? Yes, it might resonate with people, but how? What's the longevity of that piece of art or music? So that's a really interesting perspective.
Speaker 1:Another thing that this makes me think about is also because there is this tension, right, on the one hand, art really doesn't have meaning just because it exists. It's the process and the person that made the art that is at least as important, if not maybe even a little bit more important, than the outcome of the artistic creative process. Right On the flip side, for sound, my question would also be for you from a functional perspective, right, if the goal is for us to create certain emotional which are, from all we know, certainly biochemical reactions in the body, if we know enough of the science, can we just trigger those same emotions? And you know it's a very philosophical question, but I would love to get your take on. Is there a line? Is it blurred? Doesn't?
Speaker 1:even exist in the first place. What's your take?
Speaker 2:Yeah, if we know the neurological experience the particular digital experience or work of art is going to create, could we eventually just take a pill or attach some sensors and activate that, and would it mean the same thing? I think, wow, that's a huge question and really, really exciting. Actually, I think and this kind of goes back to AI in general the human intention is really important in anything that's communicative, you know, whether it's a text exchange or a work of art. And one thing that the AI cannot have, even though it can fake it very well, is real intention, real feeling, real understanding of what led to all those experiences that caused those people to write those songs that are then sometimes very recognizably, slightly rehashed. We should come back to that in a second.
Speaker 2:The actual training, I think, is different with music than it is with other data, because I just think there's less of it even though there's a lot of music in the world. But you can often recognize the specific voice, not voice the specific style of artists, sometimes a voice very, very clearly, whether it's James Brown, weather Report, herbie. There have been several things where I heard it's James Brown, weather Report, herbie. You know there have been several things when I where I heard it and I thought, wait, I've heard that recording before and it's almost like someone taking samples of that and remixing it in a very sophisticated way. So I definitely have a problem with the training, the way the training is done with AI for music and um, um, you know it's just, I'm all for remixing and for people influencing and for for free use of, you know, the public, public domain, but when you hear how powerful that is, you know you're going to do that and actually the best songs that I've heard from AI have been, have been those wait a second.
Speaker 2:Is that James Brown? Wait a second is that Wayne Shorter from weather report, if you know, for like jazz rock fusion prompt. I heard at one point it's like no, but that is. Those are weather report, those that that sound can literally be traced back digitally and and analogically back to specific performances by artists. I know, I mean, there's just no doubt about it.
Speaker 1:And people who.
Speaker 2:It's a visceral thing. So it's repeating and remixing at a very, very sophisticated, deep level, to the point where I think our laws and our ways of thinking about copyright have probably got to change. And I'm not original in saying that, but what I did. I want to circle back to the idea of intention for a second, if that's okay.
Speaker 2:Because with music, I think that's really important. If you think about text bots that can generate, or you know, a digital boyfriend or digital girlfriend, let's say, or avatar, some of those are very sophisticated and some people are actually getting really attached to them emotionally because they, statistically, they say all the right things, but that's not the same right. I mean, it's such a big subject and I don't want to take away the meaning and the value for someone who is experiencing something meaningful or even helpful from, like a you know, a digital therapist or something. There are positive outcomes and good use cases, I think, but it's also really important to always remember that your digital boyfriend is not a real boyfriend. There's not a person there who loves you or or who understands you. It is a very elaborate fake out in some ways.
Speaker 1:It doesn't mean it can't be meaningful.
Speaker 2:But when you apply that to music you have to think about the intention to you know, and and what? Um, there's a certain kind of authenticity I don't like to use that word too much because it's overused but a certain kind of humanity and just deep authenticity that you can feel when a real person is behind those sounds. Even if it's mediated through studio technology and digital technology, and even if it's, like in some cases, very even if they're programming a drum machine or something you know, people have personal ways of doing that convey their way of being in time and space and their fingerprint of being in the world and they're into reality in a way that people, that other people can perceive. You call it soul, call it authenticity, emotional valence, whatever, but we can feel that you know, and the more experienced of a listener or a participant in a certain culture or tradition you are, the more you can sense that.
Speaker 2:Now when, when I listen to some AI music and I'm not an expert on this, and just let me I just want to say there are some very exciting creative applications, right. But let's bracket that off for a minute, because there's this interesting phenomenon I've had when at first you hear something and it sounds pretty good, you know, like a country song or whatever it is. You know it's always very genre based, because that's how these things work. It's never like something new and wholly like wow, I never, I've never heard anything like that before Because, by definition, you have heard a lot like that, right, that's the whole point and in a way, it's kind of like searching for all the lowest common denominators and putting them together. But when you, when you actually then so you can have this effect we're like wow, that's obviously a good piece of music or an interesting, a competent piece of music, right. But then, like ai visual art, if you start to really listen to it, I mean, it's good for background, it might be good for marketing, it might be good for certain kinds of like generic uses where people aren't listening closely. But when you actually stop and listen to it, instead of just kind of getting an impression, something's off often, not always.
Speaker 2:I've heard a couple that that that really weren't lately, so maybe it's getting better, but there's, you know, it's like having six toes on on somebody's foot or two, or a table with six late. You know things that don't, that don't work in the real world but that ai can't understand. All those things that we find interesting or laugh at a little bit in AI visual art, but they are really important, I think, because they show that these are being. The surface might look the same, but the process that goes into that is totally different and the idea of understanding an intent is not there. Some people might disagree with me and think that machines can or that there's no difference whether it, if it looks like it's there, who cares if it's really there or not? I would beg to differ. I think it does matter if the intention is really there. What do you think?
Speaker 1:I'm with you, and one of the reasons I even started having these conversations on a podcast is because there's one very dominant narrative that's certainly driven the hype and dismissing often any sort of critical questioning or thinking as kind of obstructionism. I think why I love what you just outlined is because, at the end of the day, we're not building technology for technology's sake, right. We're building technology so that we can enrich human lives, right, and that we can relate to each other and that we can live in a world and on a planet that we all benefit from. Yes, of course, the technology is going to get better, but it only gets better to the extent to which we program it to be better or use it to do good things, and I think you did a great job outlining how we might need to discern and that doesn't mean that everything is per se bad or good, it's actually the nuance and decide when you intentionally might use generative AI tools, be that to create visuals, pictures, photos inspiration.
Speaker 2:Whatever, I think it can be great for inspiration inspiration, whatever.
Speaker 1:I think inspiration right for inspiration, yeah, but I think, infusing it then with intention and also taking the time to actually go through some form of a creative process and not just, you know, pushing a button and taking whatever the first thing is.
Speaker 1:Then it spits out and say this is now good, right, or of good quality, or of good intent, and so, yeah, I I think I think there's extremely well put, thought provoking and maybe also a good segue to one of my last few questions that I had for you, because you already already mentioned this earlier our conversation about neuroaesthetics that we had a while back, something we're both very interested in the field, that it still feels like very nascent, because it basically blends our various disciplines that make aesthetic choices with our current and hopefully future understanding of neuroscience and how those aesthetic choices impact the body and, in turn, the human perception and cognition. So what's your particular interest in neuroaesthetics and why do you think it's going to become a critical part of the toolkit for anyone who is either making things or even, I would say, leading companies? Because I think it's going to become a really big part of just how teams function too.
Speaker 2:Yeah, yeah, I almost think it should be part of everybody, um, of everybody's training and toolkit, just like communication skills, literacy, you know it's. It's a kind of literacy, it's like brain literacy or or a neurological literacy. For me personally, understanding the brain science behind sensory experience, I think, has helped me be a a much better designer in one very like, practical, demonstrable way. It helped me create the sound and sensory paradigm that I created at Microsoft for designing technology. Sound, which had, was one thing that helped to change the direction of sound, of sound design for for digital experiences. Not the only thing, it was part of a conversation with the whole discipline, but that was kind of my contribution. And one reason I left Microsoft was so I could develop that into other things. And one reason I left Microsoft was so I could develop that into other things and I'm still finding new ways to really fully explore that. For me, you know, as a performer and a composer, I had to learn my an understanding of the brain science behind the experiences.
Speaker 2:Now that we have, you know, in the last 10 to 15 years, there's just a lot more research available made possible by, especially by new imaging techniques, especially fMRI, functional magnetic resonance imaging, and some other things that actually make it much easier to study. So there's just been incredible research, some of it in a lab, some of it a little more real world, that artists can use to understand more deeply how the structure and the meaning of the experience of art, which to me is very exciting, whether you consciously use that or not, and I don't think you have to understand that to be a good artist. But I think it's one more modern tool and and maybe great artists have always understood that at some level, a lot of times it it it maybe confirms intuitions or it's not. It's not like it takes the mystery away from creativity or or the spiritual aspects or or anything like that, and if anything, I think it adds to the more nuanced understanding of of how they work.
Speaker 2:But when you can see that, like certain patterns of sound, say, for example, create very specific patterns in people's brains, even though those people's brains are very different, their experience has been very different, and I was really kind of perceptual and experiential pattern in the brain through the sensory pathways and the different kinds of brain networks that process sound, I thought that was too simplistic, but it's not. It's actually more complicated than that, obviously, but realizing that has really kind of changed the direction of my work with the sound and sensory paradigm into some things that I hope to be doing in the future that leverage that, those ideas and that research and the understanding of the biological foundations of musicality, but also multi-sensory experience in general, to create experiences that are more beneficial for people's mental health and possibly even new kinds of precision therapies. That's kind of the journey I'm on right now and we'll see where that goes in the next few years, but I'm really excited about the potential there.
Speaker 1:That's really great, and one thing that, you know, this just brought up for me was really, I think, as you said, understanding neuroaesthetics does not take away from the magic, but I think it is crucial for designers and creative leaders to own up to the responsibility and accountability that comes with knowing what, in the best and in the worst case, the effects of the work can be right, and I think that's something that is incredibly important, especially as we're rolling out more technology at scale that can impact people.
Speaker 1:You mentioned already earlier, you know, these AI boyfriend-girlfriend situations where people actually have been harmed emotionally and physically right, because we did not put in guardrails, right, and I think the same is true for any sensory design.
Speaker 1:And, just more broadly, I think, our awareness of the impact that technology has as we're putting it out into the world, and that responsibility has to develop, and the understanding of that responsibility has to develop at the same time as we're rolling out this technology. So to me, that's been a real focus, and I actually remember one of my students who studied, actually, art and neuroscience at Princeton and in one of her papers she was specifically looking at the effects of social media scrolling on, you know, on the brain and, right after, basically deleted all her social media apps from the phone. And it's, you know, it's. I think the deepening understanding of the effects that the technology we make has on human bodies is incredibly important. As much as the technology is exciting, I think it is also as important to you know balance the benefits and the risks and at least put guardrails in place so that we, to your point earlier, don't do harm in the best case, right, and I avoid as much harm being caused as possible. Yeah.
Speaker 2:Yeah, you just made me realize something that might be too simple, but it just sounded really profound to me when I was the way you were saying it and then you kind of inspired this thought that, like the experiences that we deliver and social media, global platforms, any kind of product experience they don't just have, you know, we just think of them as as happening outside of us. Our bodies are these bounded things in space. These products are, are things that we engage with externally. We don't right. I mean, where they really happen is inside us and in a way, that's obvious, but it's so obvious that it gets lost, I think.
Speaker 2:And what neuroaesthetics and thinking about the biological foundations of sensory experience do for me personally and I wish we could share this more broadly it changes how you think about that. So it literally will change your design decisions. Right, it changes the calculus. It allows you to make much more effective designs and to tune things or to tune people. You could reverse it a little bit. Our experience is actually tuned. You know my, my music. Ideally, it plays people's brains in a very pleasurable way, you know. I mean the brain is actually structured in a musical way, so that makes it kind of easy for me. But but these sensory experiences tune, tune our bodies, tune our brains and and it's not a small thing, right, it's not. It's not just kind of this abstract little thing. Oh, yes, you know, yeah, I understand, it's my whole body. When you're immersed in it, it is your whole body all the time. When you multiply that by at a societal level public health, global platform level, to kind of bring it back to where we started those impacts for society are huge and, yeah, there are lots of implications and I think it's incredibly inspiring, but also, again, very humbling and a little scary.
Speaker 2:Frankly, you know, I've put some sounds out there that when I heard In the Wild I was like, oh, my God, you know we need to change that fast. Unfortunately, I don't think we could because you know how global platforms go, but luckily, I don't think, you know, I don't think anyone died from it, but it didn't help, right? You know it was songs, sounds that were too long or too annoying, or things that sounded you know, I'll just share all my mistakes happily things that sounded great in one context that when you actually heard them in the real world context, it didn't sound right at all. And we're just going to annoy people to the point where I actually said I hope people will just turn off their sounds until I fix this, you know, or until we can.
Speaker 2:Anyway, I could give more context for that, but but you know what I'm saying. It's, it can go both ways. It's a blessing and a curse, right?
Speaker 1:It really is. It really is. Before we wrap, we've talked about a range of topics that all to me make me wonder where do you see the next 10, 15 years go? And you can take this question either very broadly or specifically around the evolution of how we, ideally, in the best way, make sense and take advantage of all of these insights and the ability to use these new technologies to really benefit most of humanity.
Speaker 2:I think one of the most important things that I hope happens and that I think is starting to happen, but that I think we probably need more kind of conceptual frameworks that I hope people will develop is that through all this kind of AI mania that's going on and excitement, hype, whatever, I hope that that leads us to a deeper understanding of what is uniquely human, you know, and that ties into the way our brain works and to you know, it's not just mechanical. In my humble opinion, the sum is more than its parts. There's a lot going on there. The brain is one of the most complex systems ever discovered and we still don't understand it, and sensory experiences activate our brains in very holistic ways. So there's so much interesting to learn there about what it is to be human and, in a way, the more a machine can do, the more it's showing us what's unique about us. Right, if it can save us as an artist, sometimes going through a lot of busy work is really important.
Speaker 2:Actually the technical parts that you have to just, you know, repetition until you get it right. You know I wouldn't trade those processes for anything, but there are interesting scenarios where we're using it to get rid of busy work for people probably really improve people's lives right and can be done with little downside. So I think understanding what's unique about humanity and also just about human consciousness if that's not too, I don't mean that in a real new age kind of woo-woo way, I mean in a very scientific rationalist but also a little bit of a spiritual, mystical way I think those things intersect. I don't think they're mutually exclusive and I think they really highlight one another.
Speaker 2:Personally, the more I learn about neuroaesthetics and the brain and how music works on our brains and bodies, the more I appreciate the spiritual aspects of it as well as the technical and scientific and sort of objective aspects of it. Because whatever it is that's going on in our brain, it creates unique phenomenological experiences. For the way you experience even though the sounds I talked about earlier might make similar patterns in everybody's brain generate certain kinds of electrical activity that are consistent across people, even though they're very different people. Your experience of it then gets filtered through your whole history. Your brain does this amazing kind of interpretive thing with it that turns it into what's meaningful for you or not meaningful for you, and that's where I think things get really, really exciting and interesting.
Speaker 1:That's a great way for us to at least wrap this conversation. Thank you so much for the time and all your insight, matt, incredibly inspiring for me. As always, and hopefully for everyone who's listening. There's a lot to be discovered. I'm actually looking forward to what you are up to next and again thank you so much for coming on the podcast. Thanks, ben. Thank you so much. All right, that's a wrap for this week's show. Thank you for listening to Poets and Thinkers. If you liked this episode, make sure you hit follow and subscribe to get the latest episodes wherever you listen to your podcast.
