91ɫ��Ƭ

• Turning speech therapy into child’s play

• Dr ChatGPT: How AI could help us live longer

• ChatGPT for medical data? Predicting the future of population health

• Hey Google, it’s time you listened closely to what our kids are saying��

• Smartphone app to screen for early signs of dementia

Neil Martin 00:06 ��
Welcome to 91ɫ��Ƭ's Engineering the Future podcast. In this episode, we'll answer a big question. Apart from simply the words we speak, how much information is hidden in our voice?��
����
Beena Ahmed 00:19 ��
When we think of speech processing, we think of speech recognition and voice ID-that is, identifying who's speaking. But speech encapsulates a lot more. It includes information about our health and well-being, such as whether we have a heart problem, a respiratory condition, depression, dementia.��
����
Neil Martin 00:42 ��
That's 91ɫ��Ƭ Associate Professor Beena Ahmed, who says that talking to technology is not just a way of controlling our TV or phone or computer, but can actually reveal vital secrets about our health and personal identity. On Engineering the Future, we speak to experts and researchers who are embracing cutting-edge ideas and pushing the boundaries of what is truly possible. Join us as we discover how world-changing action starts with fearless thinking on Engineering the Future of Speech Processing. ��
��
Hello, and welcome to Engineering the Future of Speech Processing. My name is Neil Martin, and I'm a journalist and STEM communicator working in the Faculty of Engineering at 91ɫ��Ƭ. Joining me today to discuss how our voices are being decoded by technology and what that means for the future is Associate Professor Beena Ahmed from 91ɫ��Ƭ. Her research focuses on applying machine learning and wearable technologies to healthcare, including sleep disorders, mental stress, and speech therapy. She develops innovative algorithms for remote monitoring, early disease detection and assistive technologies for people with speech impairments and chronic health conditions. Welcome, Beena.��
����
Beena Ahmed 02:09 ��
Thank you, Neil, for having me.��
����
Neil Martin 02:11 ��
So I might jump straight into this one with a question, so that our listeners get a good understanding of what we're going to be talking about today, straight from the expert. What exactly is speech processing?��
����
Beena Ahmed 02:25 ��
Speech processing is trying to decode the information present in what we say, what we communicate, and so when I say what we say, I'm not referring to just the letters that are decoded from our speech, but also things like how we're feeling, the environment we're speaking in, including removing background noise, trying to understand the emotions we are trying to convey, and also trying to decode even simple things like what language we are speaking in.��
����
Neil Martin 03:08 ��
Where would people have maybe come across speech processing already in their lives? Maybe they don't realise it's happening.��
����
Beena Ahmed 03:14 ��
Well, speech processing has been around for quite a while. The oldest would have been the talkies that we had, you know, where movies where you actually had sound in a movie that was a huge thing, and we were actually using a microphone, just like we're using in this podcast, to record a speech signal, and then translating it into something electronically that can be superimposed over a picture. So that's the original, you know. And speech processing - we're going to be using it here to enhance the audio quality of the speech. Any noises that we produce are going to be removed out. So anytime we make a mobile call or a phone call, we're using speech. The speech signal is converted into a series of bits and sent over copper or a mobile phone encoded so that there's no error. And the one that people now most associate with speech processing is voice assistants. So whenever we talk to Siri or Google and tell it to give us directions to the closest restaurant.��
����
Neil Martin 04:25 ��
People might also associate with ChatGPT. Would that be correct, or is that a different field?��
����
Beena Ahmed 04:33 ��
No, in ChatGPT, when we use the voice to enter in our input, we're using speech processing that is converted, like all the other applications, into text, which then ChatGPT works with.��
����
Neil Martin 04:46 ��
Right? Is speech hard to decode?��
����
Beena Ahmed 04:50 ��
Yes, it is because there's just too much variability. I mean, even here, the two of us, you sound very different from me. It's not just how we sound; it's how we talk as well, too. Our accents, the way we produce different sounds - there's just a lot of variability. Even an individual can have variability from one day to another day, one time of the day to another time of the day.��
����
Neil Martin 05:19 ��
If you get a bit tired towards the end of the day.��
����
Beena Ahmed 05:21 ��
Exactly - slurred speech, you know, all that sort of thing. So it's not as if we're writing something and/or typing something, you know.��
����
Neil Martin 05:30 ��
Does that make it harder for the processing side of things to happen? Like you said, if you're writing text, it's sort of generally going to be the same, but where you have these differences individually and then across populations, does that make it really complicated for the computers to be doing that processing?��
����
Beena Ahmed 05:48 ��
Yes, it is quite complicated because of that variability. I mean, I'm sure somebody from the NLP field might disagree with me, but yes, because you first have to decode that speech signal, right? It's converted electrically, but what message is being conveyed in that? And the other, aside from the variability, there's also a limit in how many recordings are available to develop models to process all that. And when I say NLP, I mean natural language processing, which is converting characters in the text to words, sentences, paragraphs, essays.��
����
Neil Martin 06:31 ��
So the computers would need a database. I mean, I think people understand this now with ChatGPT. It's been around for a little while. People have got to know, kind of, maybe not in massive detail how it works, but it's using a database of information to be able to process that text.��
����
Beena Ahmed 06:49 ��
Yes.��
����
Neil Martin 06:49 ��
Your point is that the computers don't have so much of a database of speech or an individual person's speech, I guess.��
����
Beena Ahmed 06:57 ��
Yes, I mean essentially how these models work is they look for patterns in whatever data that you are working with. With text, thanks to the internet, we have a lot of text available that people can access, create databases for, and look for those patents. And then, because there's less variability in that, you need less data to develop the models. With speech, A, it takes larger data sizes to record an audio file compared to a text file. B, you need a lot more data to capture all the patterns that you want to model, and those don't exist. And C, people sometimes don't want to share their speech, right? Because it inherently gives away our identity. It's a private piece of us. Whereas with text, we're less reluctant to be...��
����
Neil Martin 07:54 ��
More anonymous as well, I guess.��
����
Beena Ahmed 07:55 ��
Yes, and so these three factors limit how much there's actually available to develop models with.��
����
Neil Martin 08:02 ��
What kind of information is hidden in our voice-the information that the computers are processing, rather than just the words that we say.��
����
Beena Ahmed 08:13 ��
So this depends on what you're actually trying to get from the speech signal. So when we're using things like Siri or ChatGPT, all those applications care about is the actual characters or the words and sentences, so they just try to remove that information, extract that information. That's all they're concerned with. But as I mentioned earlier, there's a lot more. So there's our identity, for example, that differentiates me from any other person. There's our health, because when we speak, we use our respiratory organs, our muscular system, and then of course the articulators of the mouth to produce to push air out of our mouth. Right. So, if something's wrong in our respiratory system, our lungs, you can pick that up from your voice. Your voice sounds hoarse when you have a cough, right? Your cardiac system, your heart - if you have a heart problem, that can actually be mirrored in our speech. If you have any issues with your articulators - your tongue and your mouth - or your airways, that also comes in. And then it's a neurological thing, right? We are actually thinking about what we are going to speak. So, if there's - if you have a stroke patient, for example, people have probably come across those, right? It impacts something in the brain, and they are unable to speak. That's an extreme example, but if you're depressed, you have another neurological condition: Parkinson's, Alzheimer's. It's going to affect the way you speak, and so yeah. When we think of speech, we're just thinking about the message that we're conveying, but it conveys our well-being, our health, our identity.��
����
Neil Martin 09:58 ��
I guess as well. You can use a word, but that word can have different connotations depending on the tone, depending on the volume. I mean, even a basic word like stop, you know, can be very forceful, or it can be quite carefully worded. So, like, how hard is that? Because obviously, the word or the phrase might still be the same, but it has a completely different meaning depending on how it's delivered.��
����
Beena Ahmed 10:27 ��
Yeah, you've raised a very good point. I mean, that increases the complexity of the thing, right? Simple thing like whether it's a statement or a question, we are there or we are there. Declarative versus a question, right? Text-wise, you would differentiate it with a question mark. Yeah. Okay. So you've got a different way to differentiate the two. But if we didn't have that question mark, right? I would need to differentiate that there was a slight increase in pitch towards the end that made it a question. And so, as you said, the stop, right? How forceful you are. So you need to be able to detect these differences in pitch or volume to differentiate that. And so some of the speech models, the speech recognition models, now accommodate for that. But because we've been providing them more and more data, right? We're providing Siri and Google with data. We're giving them our audio to develop models to do that. But not everything is captured in that. ��
��
Neil Martin 11:28
Is it easy to explain what is actually happening behind the scenes when these systems get this audio to then turn it into something that the machine can understand and interpret, or does it get very technical very quickly? ��
����
Beena Ahmed 11:45 ��
It gets quite technical. But I'll try and give a very, very simplistic explanation.��
��
Neil Martin 11:50��
Thank you, thank you. ��
��
Beena Ahmed 11:51��
So essentially, when we're speaking, we're stringing together sounds into words, and then those words are formed into sentences or paragraphs. The part about going into words, sentences, and paragraphs - that's called natural language processing. So that's what the text people who work with text tend to focus on. In speech, what we're doing - we're trying to break down every word that we're saying. We look at small instances in time and try and decode what sound is being said and what character represents that sound. So the word "sound" is S-O-U-N-D. So we're trying to detect the individual sounds and map them to the corresponding characters. And so that's where we have to do the processing, so the microphone converts it into an electrical signal, and then we look at things like the pitch, the amplitude, and then derive more features from these more complex features, the tone, the things called formants, and we represent them in more complex ways that differentiate things like a pa from a ba, and then once we have those characters, we then make words from them.��
����
Neil Martin 13:12 ��
It sounds simple, but also very complex at the same time. You've done a good job of explaining it, but I'm sure it's a lot more complicated. And while you were speaking there, I was thinking that the way you say a word is different to the way I say a word. My background is from the UK. I think you're from America is it?��
����
Beena Ahmed 13:32 ��
I have an American accent. Yes.��
����
Neil Martin 13:34 ��
So, how does the processing system deal with accents? I guess, or even, I guess, then foreign languages, which are completely different again.��
����
Beena Ahmed 13:44 ��
Very good point. I mean, accents for those who've been listening when Siri first came out in Australia, it had a hard time understanding anybody Australian. There's some people still with strong Australian accents who struggle with Siri or Google, and that's because what happens is these, as we've talked about with data, right at the start, these models are trained with this volume of data that is available. So they basically, for speech recognition, they'll have all these possible ways of saying S, okay, and they'll say, okay, if you have any of these sounds, it's an S. If it's any of these Ds pronounced differently - you can have a hard D, you can have a soft D - it's a D. Now, if something is not present in that data that the model has been trained on, it's not going to recognise it. So an Australian example is the R dropping. So Americans will say car. So the R is quite strong. Australians will say ‘car’. You know, I'm still not good with an Australian accent, but that's as close. ��
����
Neil Martin 14:53 ��
I’m not even going to try.��
����
Beena Ahmed 14:54 ��
That’s as close as I can get. But that if that doesn't exist in the training data, as we call it, it's not going to recognise the car, and so we're getting, we're giving our audio more and more to these companies. So the models are becoming more robust, as the word is used, and so they're getting better with English. Okay, so they started with American, then went to British and all the other Australians, then they added other languages. But if you speak a language which is not a mainstream language, okay, even some mainstream languages struggle, like Mandarin, for example. Resources for Mandarin, for example - more people speak Mandarin than English - are more limited simply because there is less Mandarin data available. The same applies to Arabic. ��
����
Neil Martin 15:41 ��
I’m thinking, even with British, even in the UK, we're famously, you know, you go two streets down and you get a completely different accent. So even just saying we're going to go to the UK and get people to speak, you would need multiple data points to cover all the different accents. Yeah,��
����
Beena Ahmed 16:02 ��
Exactly. And if you're a non-native speaker of that language, so non-native English speaker, they're going to bring the way they say certain sounds in their language to English, and so they're going to change the way they say an English word, and so that's not going to be in the database. So that's why non-native speakers struggle as well.��
����
Neil Martin 16:24 ��
Is there a danger that we might, with these systems and people having to change the way they speak to be able to be understood, that you maybe get rid of the richness of language and accents if you have to speak in a different way to be understood by the computer, you might drop your accent a little bit.��
����
Beena Ahmed 16:43 ��
Yes, it is a danger, and linguists are concerned about that in that sense that certain languages or accents are dying out because we're stereotyping, and it's not just because of using voice assistants; it's also due to exposure, right? Where, for example, even I've spoken to linguists here in Australia who are concerned that Australian English is starting to drift more American because of our exposure to American media.��
����
Neil Martin 17:16 ��
Everything becomes a bit more generic and globalised.��
����
Beena Ahmed 17:19 ��
Yes, a lot of languages are dying out because of that, and now, for example, we're getting synthesised speech, so voiceovers, right? And predominantly they are American, whereas previously you would have somebody actually record something for the Australian audience.��
����
Neil Martin 17:39 ��
You mentioned Siri and Alexa, and I guess most people would know what they are. But how else is speech processing already being utilised? Is there something sort of surprising or less obvious that you think people are not so aware of where it's being implemented?��
����
Beena Ahmed 17:57 ��
So I talked about improving audio quality, people don't realise that when you're making a recording, how much is done to clean it up, so it's presented in whatever form you're listening to - a podcast, a video, a clip. And you have YouTubers, right? Vloggers making videos on the street, and there's enormous background noise around them, and when you watch it on YouTube, it's all cleaned up. Speech processing is not just the actual microphone that you're using, but then the noise cancellation or speech enhancement. So that's a very low level, but then applications start coming up where we're using things that are present in our speech, so our markers of our voice identify who we are, and so we're getting applications. The ATO, for example, provides us the option to use our voice for voice authentication. Banks are using that too, so voice ID is important, and another one is speech synthesis, which we're now more and more seeing. So voiceovers - we're all getting used to the fact that clips on YouTube are somebody that we're watching. Somebody's not actually speaking. It's synthesised speech. People are using it in commentary as well, and ads. Those are all applications where speech processing is being used.��
����
Neil Martin 19:27 ��
And I guess there's a lot of artificial intelligence tied in with all of that as well.��
����
Beena Ahmed 19:32 ��
Yes, yes. Artificial intelligence is a huge field, and I'm sure you'll find people who can tell you about this, but AI is not just the language models that we're using to generate text. Speech processing is part of that AI field. It's just a different modality, speech.��
����
Neil Martin 19:55 ��
And going back to where you said about using your voice as an ID. I guess that means that your voice is unique. You're kind of quite confident that if somebody says a word into my bank account, it's not going to think it's me.��
����
Beena Ahmed 20:11 ��
Okay, so there are two points there. Yes, your voice is unique, and nobody can sound exactly like you. They can try and sound like you, okay? Children can sound a bit like their parents or siblings can sound similar. People can imitate you, okay? We have all those Elvis impersonators, for example, but there's still, if you are trained, you will be able to differentiate between Elvis and an Elvis impersonator. Now we have to come up with systems that can detect these fakes, and so that whole builds up a whole avenue, you know, spoofing and then anti-spoofing. So I mean, if we if you go to some of the major conferences, there was one major one on speech technology, , which we're actually having in Sydney later this year in October, 1st week of October, end of September. So we'll have experts in this area. So, for example, speech privacy was a field that did not exist or very minimal work on it 20 or 30 years ago. Now it's very important because people want to anonymise speech, spoofing, also anti-spoofing. When we get together in these conferences, as you said, researchers are bringing new ideas and how to deal with this. So it's, I guess, the evolution of technology. ��
����
Neil Martin 21:30 ��
I need to take you back to something that you mentioned before, and it sounded really interesting: the way that your voice and your speech can be used as a diagnostic tool in health. That's not something that I'm necessarily have heard a lot about, but it sounds really interesting. And do you see that developing much more over the next 20 or 30 years?��
����
Beena Ahmed 21:52 ��
Oh, definitely. I think that's where most of the progress is going to happen, as I mentioned. So some of my work is trying to detect the presence of early onset of dementia in speech, or actually pre-dementia. So you want to identify if someone is at risk of dementia, so that you can do an intervention to delay the onset of dementia.��
����
Neil Martin 22:13 ��
And how does that show itself up? I guess in the speech that you're processing.��
����
Beena Ahmed 22:18 ��
So speech is where dementia actually shows up, one of the earliest markers of dementia. When we think of dementia, we think, "Oh, they forgot to say certain words or things like that”. Vocabulary decreases, but it first impacts how we actually speak. And a good analogy: when you're not confident about something, you tend to pause, give more thought about what you're saying. So, if somebody is under increased cognitive load, then the way they speak will change. There will be more pauses between the words. Each word they will take longer, even if it's microseconds. You know, it is a change.��
����
Neil Martin 22:59 ��
And that would be relatively imperceptible, I guess, to a human. You know, if you're talking microseconds, a human is not going to pick up on that.��
����
Beena Ahmed 23:08 ��
Exactly, and so that's where processing comes in. We look, we measure those little things, and then we can track how a person is changing over time.��
����
Neil Martin 23:19 ��
I was going to say you would need the record of what they used to sound like when they were healthy, for want of a better word, to then compare and to notice those deterioration in speech, I guess.��
����
Beena Ahmed 23:34 ��
Yes, ideally that's what you would want. It goes back to that point we were talking about about the variability, right? Especially when it comes to markers of health, there's just so much variability between people, and there's very actually very limited audio available to work with. So there's a lot more audio for applications like speech recognition, but when it comes to health, we're even more reluctant to share our audio. So that makes it really hard to develop these models, But yes, ideally, what you'd want is you'd want your device listening to you every day, or you say something to your device every day, and it tracks changes. At some point it may say, "Okay, maybe you should go and see a doctor”.��
����
Neil Martin 24:19 ��
Do you think that if we look to the future, if we look 20 or 30 years ahead, do you think that that's something that would be much more common that the health system, the health experts would be saying we need this in place? And I guess I might link it to kind of devices that people wear now. There's smartwatches, it might be monitoring their heartbeat, we've kind of got used to having these devices that are monitoring us. Do you think the health experts would say, "Well, we need that audio as well because that's also giving us some really good information”.��
����
Beena Ahmed 24:53 ��
Yes, I think it will be. Just as you mentioned with those devices, the wearables that we're now wearing. The audio will also start to be captured. Maybe we won't capture audio 24 hours a day because A) ��that's a lot of data, and some of it is going to have private information in it. But it could be that we just get samples at some point during the day, and we do that. And health professionals now realise the value of audio, so they are getting interested in this. We still have to navigate the data sharing part, right? Because speech inherently is private - you could be saying something you don't want to share.��
����
Neil Martin 25:33 ��
Are there any other areas where you think speech processing will really kind of change the landscape or bring some really good benefits? I'm thinking, I don't know, maybe in education or learning.��
����
Beena Ahmed 25:48 ��
Definitely. I mean, right now we're used to when we're interacting with technology, we're used to typing. We're getting used to now with ChatGPT talking to it, and so that will filter into education, so you could have an interactive tutor, for example. You speak to it, it listens, and then it speaks back to you because we've got speech synthesis now as well, and so it could actually remove a reliance on screens, and so you could actually have a book instead and be talking to your device and reading a book at the same time,��
����
Neil Martin 26:22 ��
Like having your own personal tutor that doesn't cost $300 an hour. Exactly. And students are��
����
Beena Ahmed 26:29 ��
Exactly. And students are using technology; they're using ChatGPT or Gemini to learn, and some of them are using voice as well.��
����
Neil Martin 26:37 ��
Would the speech processing be able to assess kind of at what level of education and knowledge a person would be at to be able to then amend the information being given back? It sounds like it sounds like you could get really quite personalised.��
����
Beena Ahmed 26:56 ��
Yes, you would integrate speech recognition with some natural language processing, and see, okay, did they use the right terms? You know, and sort of figure out, okay, yes, now they need to go to level one, level two, level three. You know, judge their answers. So you would have that integration of speech recognition systems with language models to develop, and people are starting to work on those. And there are systems that are coming out.��
����
Neil Martin 27:26 ��
And I guess that could be a benefit to a wide range of people, not least maybe people in remote areas that don't have access to very good, you know, education standards. Or, or you know, I think there are teacher shortages in most western countries, you know, to be able to have a system that can assist with teaching and education.��
����
Beena Ahmed 27:46 ��
Yes, yes, definitely. Technology is making education more equitable.��
����
Neil Martin 27:53 ��
You talk about equity there, and that's made me think of something else with regards to speech processing to be able to help people whose speech maybe isn't as good. Are there benefits there, or the applications and research going on? How do you see that developing?��
����
Beena Ahmed 28:10 ��
Yeah, and so that's of particular interest to me, and I'm glad you brought it up. Most of the focus to date has been on providing technology tools for people who have typical speech, okay, or easy to understand. Whereas if we think about it, I could probably get away without using a voice assistant. Whereas somebody who has a speech disorder of some sort or a physical disability of some sort together would really appreciate an assistant to help them turn the lights on, things like that. That is where the need is. But these systems are very difficult to develop because, again, the audio becomes even more variable. But I think that is where technology is now moving to try and help people with speech disorders, things like dysarthria. I spoke about stroke patients to help in their rehab. Some of my research is with children with speech disorders, providing them therapy assessments so that they can actually improve, not just interact with, and use an assistant, but also improve their speech.��
����
Neil Martin 29:28 ��
If somebody had a level of speech which was difficult to understand, let's say, would a speech processing system then be able to interpret them better and to kind of help them communicate to the outside world - is that a goal?��
����
Beena Ahmed 29:47 ��
That's the goal. It's not there yet, but keep in mind that humans also struggle to understand them. So we're working towards that aim. We have a lot of people who struggle to communicate. You hear about people who just end up staying inside at their home because they're unable to communicate with the outside world, and so yeah, the idea is to allow them to become an active part of society.��
����
Neil Martin 30:14 ��
And also, like you said, to be able to interact with objects or with other technology - that's going to be a benefit to them.��
����
Beena Ahmed 30:21 ��
Yes, definitely. And I mean, in today's world, I think the downside of having all this generative AI is that we're going to be craving these in-person conversations even more, because they're going to be more authentic than a video call or all of that, and so we'll be wanting to be understood when we speak, and so there'll be a lot of effort put into applications that help us improve, not just for typical speakers to train how you know how to give a podcast, for example, but also for people who struggle to communicate as well.��
����
Neil Martin 31:00 ��
And if we look ahead that 20 or 30 years with the advancement of speech processing, overall, how do you think it's going to change the way we communicate with each other, with technology, with our peers?��
����
Beena Ahmed 31:15 ��
That's a very interesting question. As I sort of alluded to, I personally think that we'll be having a lot more in-person conversations because we're going to be inundated with generative, you know, or AI-generated video conversations, and so we'll want that real connection. And I think that's already happening.��
����
Neil Martin 31:38 ��
Do you think people are getting wary? I mean, like you said, in terms of spoofing and faking, is that where you're coming from? That you think people might be like, well, unless I'm actually in the room with this person, how can I be absolutely sure it's them? I'd much rather speak to them face to face, then I can be sure that.��
����
Beena Ahmed 31:58 ��
Part of it is that. Okay, so for example, if you're interviewing a job candidate, right, you're not sure if you're on a video call, is that how they actually sound? Okay, that's part of it. But the other thing is that when you have too much of one thing, you want something different, right? So, if you're getting too much of tech-generated speech or video and audio from technology, you actually will want the real thing.��
����
Neil Martin 32:26 ��
And you talked about the conference there at 91ɫ��Ƭ. What are your hopes and aims from that? How does that then project through to the future? What would be your big dream? Maybe just one thing or a general benefit from speech processing over the next 20 or 30 years. ��
����
Beena Ahmed 32:45 ��
Our conference theme for Interspeech 2026 is equity and diversity. So, our focus is trying to raise awareness of the work that's being done in areas that make speech technology more equitable. So, things like health, speech therapy, voice assistance for people with speech disorders, non-native language processing, pronunciation and accent training and modification, areas that are not highlighted, you know, and so that's what our hope is. We're trying to bring in people who work in these areas to highlight these topics.��
����
Neil Martin 33:27 ��
It definitely sounds like, as this field keeps evolving, our voices won't just be how we communicate, but also how technology understands us. Associate Professor Beena Ahmed, once again, many thanks for joining me. It's been absolutely fascinating speaking with you.��
����
Beena Ahmed 33:44 ��
Thank you very much for having me, Neil. I really enjoyed the conversation.��
����
Neil Martin 33:47 ��
Unfortunately, that's all we've got time for right now. Thank you for listening. I've been Neil Martin, and I hope you'll join me again soon for the next episode of our Engineering the Future series. You've been listening to 91ɫ��Ƭ's Engineering the Future podcast, highlighting the people and ideas creating real-world impact, and enabling progress for all. Don't forget to subscribe to stay updated on upcoming episodes. Check out our show notes for details on in-person events, panel discussions, and more fascinating insights into the future of engineering.��