Do phones use voice recognition technology to eavesdrop?

[Stephen Tashi]

TL;DR Summary

Do smart phones use voice recognition technology to eavesdrop on conversations?

I've seen examples where a smart phone apparently used voice recognition technology to eavesdrop on conversations and brought up things mentioned as suggested terms in Google search. I mean conversations that were not phone calls - just ordinary conversations. For example, suppose I am talking (in person) to someone about "baking pans". Afterwards the person takes his phone out of his pocket and begins a Google search by typing "ba...", the search brings up a list of possible search terms with "baking pans" at the head of the list.

I've seen this happen with Android phones. Do phones actually listen to conversations? Or is there some round-about explanation for this phenomenon? Perhaps when the whole world is interested in baking pans, I'm more likely to be talking to a friend about them.

 

[anorlunda]

Any device with a microphone or a camera can potentially bug you and your conversations. How do you know if they are turned on all the time?

Some devices have a LED that they claim lights if the camera is on. Do you choose to believe that claim? At least with a camera, you can cover it with a piece of tape.

I remember a scandal from 1994 when Sun workstations were discovered to have shipped with the microphone open and unprotected. Since then, nothing has changed.

The only way to be secure is to track traffic to/from your devices looking for confidential data being exfiltrated. And oh yes, strictly control physical access to all your equipment.

 

[russ_watters]

Summary: Do smart phones use voice recognition technology to eavesdrop on conversations?

I've seen examples where a smart phone apparently used voice recognition technology to eavesdrop on conversations and brought up things mentioned as suggested terms in Google search.

They aren't supposed to record or use what they hear when they are not activated, but there have been "bugs" that and mis-hearing of the activation word when not active. So I wouldn't trust them. The only way to ensure your conversations are private is to turn these devices off entirely.

 

[DaveC426913]

I asked the exact same question a while back.

https://www.physicsforums.com/threads/is-my-phone-actually-listening-in-on-me.960553/
My friends and I did a (one-time) experiment:

TL;DR:

We left his phone on the table and started talking about a random subject - in this case, Nike. We just inserted "I really could use a pair of Nikes" and other such positive comments into our normal conversation.

After about ten minutes, he searched for "Where can I buy ..."

The result was negative.
That doesn't prove it isn't happening, but at least it shows it isn't always happening.

 

[Klystron]

For sake of discussion we can break the original question into discrete functions.

1. the phone microphones detect audio signals from background noise.
2. software recognizes certain audio as human speech.
3. given a wide enough sample of speech, a language is identified.
4. within a recognized language the speech is translated/decoded.
5. given a deep enough speech sample, the speaker is recognized.

1 and 2 run more or less continuously constituting 'speech activation'.
3 and 4 may run offline from the phone and constitute 'spoken language translation'.
5 is referred to as 'Speaker ID' while 1-5 constitute 'voice recognition'.

Dividing these steps already seems quaint given modern speech technology but each step was considered a breakthrough in its time. Software can recognize a speaker without knowing the spoken language or even parsing words. Spoken language translators can adjust to different speakers without identifying the speaker.

In combination any voice activated device or essentially any wi-fi enabled device with microphones can potentially be used to eavesdrop. BTW the popular notion of disguising speech by flushing toilets, playing music or running water is wishful thinking easily defeated by signal processing. See items 1 and 2 above.

 

[HankDorsett]

On Android phones there are ways you can stop it from listening to you all the time. Most obvious is not setting up the voice assistant that allows you to control your phone with voice commands. You should also move the Google search bar off of your home screen. When it's on the home screen it's listening for the OK Google. If you move it to another screen it will only listen when you are on that screen. Fortunately if you are using these voice commands the phone doesn't save anyting or send it anywhere. However, any Google voice searches are sent to Google and saved under your account. You can change account settings to prevent voice and even text searches from being saved.

Now for a bit of a story on Alexa. When it originally came out it recorded and saved just about everything to the internet. These recordings were able to be used as evidence for a domestic violence case that ended in murder.

 

[anorlunda]

On Android phones there are ways you can stop it from listening to you all the time. Most obvious is not setting up the voice assistant that allows you to control your phone with voice commands.

That assumes the voice assistant is the only possible culprit. That sounds unjustified to me.

 

[HankDorsett]

That assumes the voice assistant is the only possible culprit. That sounds unjustified to me.

I was limiting my response to the built-in OS capabilities on modern phones. With all of the Security Experts, public and private, searching for exploits, something like this would have been found and reported on. That being said, I can think of three ways this could happen.

New app download from the Play Store: although Google reviews the code of new apps before they allow it on the Play Store exploits have been missed. Most of these exploits are found after the Security Experts review on after they hit the Play Store.

Apps that you purposely download enabling this feature through a website: there are apps available that allow you to sign on to a website and activate certain features on your phone, microphone, camera, location and such. It is possible for someone, hackers and administrators of the website, to access your account on these websites and activate these features without your knowledge and approval.

Hacking your phone: first, it's not like the movies. You can't just sit in a room and access someone's phone like they show in the movies. In order to hack the phone you need to install software on it. As long as you didn't activate developer mode and allow any third-party software to be installed you should be safe. If these are not activated someone would need physical access to your phone to install something.

If you are concerned about this and technically proficient you can see if anything like this is happening on your phone. You could install a network sniffer on your own network and then analyze Network packets to see if anything out of the ordinary is going on.

 

[DavidSnider]

Hacking your phone: first, it's not like the movies. You can't just sit in a room and access someone's phone like they show in the movies. In order to hack the phone you need to install software on it. As long as you didn't activate developer mode and allow any third-party software to be installed you should be safe. If these are not activated someone would need physical access to your phone to install something.

This is not true. There are many many other possible attack vectors into a persons phone that do not require you to install new software.

 

[HankDorsett]

This is not true. There are many many other possible attack vectors into a persons phone that do not require you to install new software.

Can you provide a list of some of these so I can update my belief?

 

[DavidSnider]

Can you provide a list of some of these so I can update my belief?

https://www.theregister.co.uk/2019/02/07/android_january_patches/

 

[HankDorsett]

https://www.theregister.co.uk/2019/02/07/android_january_patches/

Thanks. The only one I'm concerned about is the PNG image exploit. I run Norton AntiVirus on my phone, you would think it would protect you against attacks like this as soon as they are known. I've searched every which way I could think of to see if Norton protected against this exploit and I couldn't find anything. I'm actually starting to wonder what Norton actually defends against on android.

 

[DavidSnider]

Thanks. The only one I'm concerned about is the PNG image exploit. I run Norton AntiVirus on my phone, you would think it would protect you against attacks like this as soon as they are known. I've searched every which way I could think of to see if Norton protected against this exploit and I couldn't find anything. I'm actually starting to wonder what Norton actually defends against on android.

Android has already patched the PNG problem, but this is just one example in an entire class of exploits that AV software is pretty much useless against. PC and cell phone hardware architectures are fundamentally insecure. We've come a long way in working around it, but it's still a major problem.

All antivirus does is look for known 'bad actors' and its not always easy to tell what is and isn't malware. It also takes a while for new threats to get into the AV database. So it won't protect you from new things. Most security professionals don't even bother to use it on their personal machines.

 

[HankDorsett]

Android has already patched the PNG problem, but this is just one example in an entire class of exploits that AV software is pretty much useless against. PC and cell phone hardware architectures are fundamentally insecure. We've come a long way in working around it, but it's still a major problem.

All antivirus does is look for known 'bad actors' and its not always easy to tell what is and isn't malware. It also takes a while for new threats to get into the AV database. So it won't protect you from new things. Most security professionals don't even bother to use it on their personal machines.

I guess in the end our best defense is most of us are not interesting enough to be listened in on.

Antivirus question for you. A year or so back I came across an anti-virus software from a relatively unknown company that claim to defend against actions rather than known attacks. At the time it was only available to corporations so I lost track of it. Any chance you know the name of the software I'm talking about? Do you have an opinion on that type of method?

 

[DavidSnider]

I guess in the end our best defense is most of us are not interesting enough to be listened in on.

Antivirus question for you. A year or so back I came across an anti-virus software from a relatively unknown company that claim to defend against actions rather than known attacks. At the time it was only available to corporations so I lost track of it. Any chance you know the name of the software I'm talking about? Do you have an opinion on that type of method?

Digital Guardian maybe? They are pretty well known though. There's a lot of stuff out there that will let you define what your sensitive data is and throw up red flags when it's being transported out of your defined safety zones or any other kind of policy you want.

That's all great, but it takes a lot of work and active maintenance which makes it not so great for home users.

There's always a trade off between security and convenience.