Do phones use voice recognition technology to eavesdrop?

  • Thread starter Stephen Tashi
  • Start date
  • Tags
    Technology
In summary, there are concerns about smart phones potentially using voice recognition technology to eavesdrop on conversations, even when they are not activated. While there are measures that can be taken to limit this possibility, such as disabling voice assistants and moving the Google search bar off the home screen, there have been instances of apps and websites being used to access microphone and other features without the user's knowledge. The best way to ensure privacy is to turn off these devices entirely and monitor for any unauthorized activity.
  • #1
Stephen Tashi
Science Advisor
7,861
1,598
TL;DR Summary
Do smart phones use voice recognition technology to eavesdrop on conversations?
I've seen examples where a smart phone apparently used voice recognition technology to eavesdrop on conversations and brought up things mentioned as suggested terms in Google search. I mean conversations that were not phone calls - just ordinary conversations. For example, suppose I am talking (in person) to someone about "baking pans". Afterwards the person takes his phone out of his pocket and begins a Google search by typing "ba...", the search brings up a list of possible search terms with "baking pans" at the head of the list.

I've seen this happen with Android phones. Do phones actually listen to conversations? Or is there some round-about explanation for this phenomenon? Perhaps when the whole world is interested in baking pans, I'm more likely to be talking to a friend about them.
 
  • Like
Likes Greg Bernhardt
Computer science news on Phys.org
  • #2
Any device with a microphone or a camera can potentially bug you and your conversations. How do you know if they are turned on all the time?

Some devices have a LED that they claim lights if the camera is on. Do you choose to believe that claim? At least with a camera, you can cover it with a piece of tape.

I remember a scandal from 1994 when Sun workstations were discovered to have shipped with the microphone open and unprotected. Since then, nothing has changed.

The only way to be secure is to track traffic to/from your devices looking for confidential data being exfiltrated. And oh yes, strictly control physical access to all your equipment.
 
  • #3
Stephen Tashi said:
Summary: Do smart phones use voice recognition technology to eavesdrop on conversations?

I've seen examples where a smart phone apparently used voice recognition technology to eavesdrop on conversations and brought up things mentioned as suggested terms in Google search.
They aren't supposed to record or use what they hear when they are not activated, but there have been "bugs" that and mis-hearing of the activation word when not active. So I wouldn't trust them. The only way to ensure your conversations are private is to turn these devices off entirely.
 
  • #4
I asked the exact same question a while back.

https://www.physicsforums.com/threads/is-my-phone-actually-listening-in-on-me.960553/
My friends and I did a (one-time) experiment:

TL;DR:

We left his phone on the table and started talking about a random subject - in this case, Nike. We just inserted "I really could use a pair of Nikes" and other such positive comments into our normal conversation.

After about ten minutes, he searched for "Where can I buy ..."

The result was negative.

That doesn't prove it isn't happening, but at least it shows it isn't always happening.
 
  • Like
Likes PeroK and russ_watters
  • #5
For sake of discussion we can break the original question into discrete functions.
  1. the phone microphones detect audio signals from background noise.
  2. software recognizes certain audio as human speech.
  3. given a wide enough sample of speech, a language is identified.
  4. within a recognized language the speech is translated/decoded.
  5. given a deep enough speech sample, the speaker is recognized.
1 and 2 run more or less continuously constituting 'speech activation'.
3 and 4 may run offline from the phone and constitute 'spoken language translation'.
5 is referred to as 'Speaker ID' while 1-5 constitute 'voice recognition'.

Dividing these steps already seems quaint given modern speech technology but each step was considered a breakthrough in its time. Software can recognize a speaker without knowing the spoken language or even parsing words. Spoken language translators can adjust to different speakers without identifying the speaker.

In combination any voice activated device or essentially any wi-fi enabled device with microphones can potentially be used to eavesdrop. BTW the popular notion of disguising speech by flushing toilets, playing music or running water is wishful thinking easily defeated by signal processing. See items 1 and 2 above.
 
Last edited:
  • Like
Likes QuantumQuest and russ_watters
  • #6
On Android phones there are ways you can stop it from listening to you all the time. Most obvious is not setting up the voice assistant that allows you to control your phone with voice commands. You should also move the Google search bar off of your home screen. When it's on the home screen it's listening for the OK Google. If you move it to another screen it will only listen when you are on that screen. Fortunately if you are using these voice commands the phone doesn't save anyting or send it anywhere. However, any Google voice searches are sent to Google and saved under your account. You can change account settings to prevent voice and even text searches from being saved.

Now for a bit of a story on Alexa. When it originally came out it recorded and saved just about everything to the internet. These recordings were able to be used as evidence for a domestic violence case that ended in murder.
 
  • #7
HankDorsett said:
On Android phones there are ways you can stop it from listening to you all the time. Most obvious is not setting up the voice assistant that allows you to control your phone with voice commands.
That assumes the voice assistant is the only possible culprit. That sounds unjustified to me.
 
  • Like
Likes russ_watters
  • #8
anorlunda said:
That assumes the voice assistant is the only possible culprit. That sounds unjustified to me.
I was limiting my response to the built-in OS capabilities on modern phones. With all of the Security Experts, public and private, searching for exploits, something like this would have been found and reported on. That being said, I can think of three ways this could happen.

New app download from the Play Store: although Google reviews the code of new apps before they allow it on the Play Store exploits have been missed. Most of these exploits are found after the Security Experts review on after they hit the Play Store.

Apps that you purposely download enabling this feature through a website: there are apps available that allow you to sign on to a website and activate certain features on your phone, microphone, camera, location and such. It is possible for someone, hackers and administrators of the website, to access your account on these websites and activate these features without your knowledge and approval.

Hacking your phone: first, it's not like the movies. You can't just sit in a room and access someone's phone like they show in the movies. In order to hack the phone you need to install software on it. As long as you didn't activate developer mode and allow any third-party software to be installed you should be safe. If these are not activated someone would need physical access to your phone to install something.

If you are concerned about this and technically proficient you can see if anything like this is happening on your phone. You could install a network sniffer on your own network and then analyze Network packets to see if anything out of the ordinary is going on.
 
  • #9
HankDorsett said:
Hacking your phone: first, it's not like the movies. You can't just sit in a room and access someone's phone like they show in the movies. In order to hack the phone you need to install software on it. As long as you didn't activate developer mode and allow any third-party software to be installed you should be safe. If these are not activated someone would need physical access to your phone to install something.

This is not true. There are many many other possible attack vectors into a persons phone that do not require you to install new software.
 
  • Like
Likes QuantumQuest and Klystron
  • #10
DavidSnider said:
This is not true. There are many many other possible attack vectors into a persons phone that do not require you to install new software.
Can you provide a list of some of these so I can update my belief?
 
  • #12
DavidSnider said:
Thanks. The only one I'm concerned about is the PNG image exploit. I run Norton AntiVirus on my phone, you would think it would protect you against attacks like this as soon as they are known. I've searched every which way I could think of to see if Norton protected against this exploit and I couldn't find anything. I'm actually starting to wonder what Norton actually defends against on android.
 
  • #13
HankDorsett said:
Thanks. The only one I'm concerned about is the PNG image exploit. I run Norton AntiVirus on my phone, you would think it would protect you against attacks like this as soon as they are known. I've searched every which way I could think of to see if Norton protected against this exploit and I couldn't find anything. I'm actually starting to wonder what Norton actually defends against on android.

Android has already patched the PNG problem, but this is just one example in an entire class of exploits that AV software is pretty much useless against. PC and cell phone hardware architectures are fundamentally insecure. We've come a long way in working around it, but it's still a major problem.

All antivirus does is look for known 'bad actors' and its not always easy to tell what is and isn't malware. It also takes a while for new threats to get into the AV database. So it won't protect you from new things. Most security professionals don't even bother to use it on their personal machines.
 
  • #14
DavidSnider said:
Android has already patched the PNG problem, but this is just one example in an entire class of exploits that AV software is pretty much useless against. PC and cell phone hardware architectures are fundamentally insecure. We've come a long way in working around it, but it's still a major problem.

All antivirus does is look for known 'bad actors' and its not always easy to tell what is and isn't malware. It also takes a while for new threats to get into the AV database. So it won't protect you from new things. Most security professionals don't even bother to use it on their personal machines.
I guess in the end our best defense is most of us are not interesting enough to be listened in on.

Antivirus question for you. A year or so back I came across an anti-virus software from a relatively unknown company that claim to defend against actions rather than known attacks. At the time it was only available to corporations so I lost track of it. Any chance you know the name of the software I'm talking about? Do you have an opinion on that type of method?
 
  • #15
HankDorsett said:
I guess in the end our best defense is most of us are not interesting enough to be listened in on.

Antivirus question for you. A year or so back I came across an anti-virus software from a relatively unknown company that claim to defend against actions rather than known attacks. At the time it was only available to corporations so I lost track of it. Any chance you know the name of the software I'm talking about? Do you have an opinion on that type of method?

Digital Guardian maybe? They are pretty well known though. There's a lot of stuff out there that will let you define what your sensitive data is and throw up red flags when it's being transported out of your defined safety zones or any other kind of policy you want.

That's all great, but it takes a lot of work and active maintenance which makes it not so great for home users.

There's always a trade off between security and convenience.
 

1. How do phones use voice recognition technology to eavesdrop?

Phones use voice recognition technology through their built-in virtual assistants, such as Siri, Google Assistant, or Alexa. These virtual assistants are constantly listening for their wake words and will only start recording when the wake word is detected. However, there have been concerns that these devices may accidentally activate and record conversations without the user's knowledge or consent.

2. Can phones record conversations without being prompted?

In theory, yes, phones can record conversations without being prompted if their virtual assistants are accidentally activated or if there is a glitch in the system. However, most phone manufacturers have strict privacy policies in place and state that recordings are only made after the wake word is detected.

3. How can I prevent my phone from eavesdropping on my conversations?

To prevent your phone from eavesdropping, you can turn off the voice recognition feature on your virtual assistant or disable the virtual assistant altogether. You can also be mindful of your surroundings and avoid saying sensitive information when your phone is nearby.

4. Is voice recognition technology used for eavesdropping by companies or governments?

There have been concerns that companies or governments may use voice recognition technology for eavesdropping purposes, but there is no concrete evidence to suggest that this is happening. In most cases, the virtual assistants are only listening for their wake words and do not transmit any data unless the wake word is detected.

5. What measures are in place to protect user privacy when using voice recognition technology on phones?

Phone manufacturers have strict privacy policies in place to protect user privacy when using voice recognition technology. This includes limiting the amount of data that is collected, requiring consent from the user before recording, and providing options to disable the feature. Additionally, users can also review and delete their voice recordings from their device or online accounts.

Similar threads

  • Computing and Technology
Replies
14
Views
1K
  • Computing and Technology
2
Replies
50
Views
8K
Replies
10
Views
2K
  • Electrical Engineering
Replies
1
Views
1K
Replies
2
Views
2K
  • Quantum Physics
Replies
4
Views
2K
Replies
3
Views
634
Replies
10
Views
2K
Replies
4
Views
3K
Back
Top