1) App stores the trailing two minutes of speech in memory.
2) If the app detects a consumption-related trigger word, the related conversation is flagged for transmission to the server.
3) Flagged audio block is converted to text. Consumption related verbs ("buy", "purchase", etc) are identified. The syntax of the sentence clearly indicates which noun is the target of a given consumption-related verb ("new car", "pool fencing")
Lots of people run network traffic sniffers to see what apps are doing. Lots of people decompile apps. Lots of people at companies leak details of bad things they are doing.
Why has nobody been able to demonstrate this beyond anecdotes about talking about swimming pools and then getting adverts for swimming pool stuff?
These are fair questions! I'm not convinced that it is happening. Nor am I convinced, as the parent seems to be, that it would be difficult to do.
edit: Having re-read my comment, I can see how it could easily be read to say "It's happening and this is how it works", whereas I intended to convey something like "It could easily be done and here's how." I have a bad habit of implying my point rather than stating it outright. I'm working on it!
I have a suspicion it’s not Facebook or Google listening in, but rather other third party apps. In fact it’s not even the third party apps but the libraries/frameworks they use to show ads.
Android shows when an app is using the microphone with a green indicator in the upper right corner -- I'm assuming iOS has something similar. How would apps get around this?
Easy: That indicator is not always on when the mic is.
Unless we're talking about an electret capsule with a physical LED wired into the supplying power rail that is switched off when the mic is not in use, you have to trust software.
And good luck with that after the patriot act. I am not implying the NSA has a microphone backdoor, but if they had and someone abused it, how would you know about it?
Listening in for keywords and only send text/audio when keywords are spoken isn't only good for ads, that would be dream of any intelligence agency. And since Snowden have been a few years.
But would you use it to show ads if you have access to such a backdoor?
That's an easy way for your backdoor to be found out. Something like this, if it exists, would be too valuable to be used en masse.
Well if I am a secret service I could either try to force google to do it and risk a leak or I could find a way that there is something in it for them that has the benefit of providing plausible deniability?
Yeah but that app was just nightmarishly bad, including an absolutely terrible approach to roll-your-own push notifications. Never attribute to malice that which can be explained by incompetence.
There will be no proof until somebody inside Apple who is in on the scam decides to grow a conscience and blow the whistle. Then they will be dismissed as a "disgruntled employee". Decompiling Siri probably will get you a lot of attention from very expensive lawyers that will make your life very interesting for a long while.
I can't even begin to tell you how many times I've been randomly having a conversation with someone, only to be alerted to the sound of the Google Assistant suddenly responding to what we're saying. Something we said was interpreted as a wake word, and then from that point on, every single thing we said was transcribed via STT, sent to Google's servers, various Google search queries were run, etc, and then the assistant responded - because it thought it was responding to a valid query and had no way of knowing otherwise. This has gotten worse with Gemini but has in no way been limited to that.
In this situation, I was alerted to this because the assistant started responding. However, I've also been in situations where I tried deliberately to talk to the assistant and it failed silently. In those situations, the UI spawns the Assistant interaction dialog, listens to what I say and then just silently closes. Sometimes this happens if there's too much background noise, for instance, and it then just re-evaluates that it wasn't a valid query at all and exits. Sometimes some background process may be frozen. Who knows if this happens before or after sending the data to the server. Sometimes the dialog lingers, waiting for the next input, and sometimes it just shuts off, leaving me (annoyingly) to have to reopen the dialog.
Putting that together, I have no idea how many times the Google Assistant has activated in my pocket, gone live, recorded stuff, sent it to Google's servers, realized it wasn't a valid query, and shut off without alerting me. I've certainly seen the Assistant dialog randomly open when looking at my phone plenty of times, which is usually a good indicator that such a thing has happened. If it silently fails in such a way that the UI doesn't respawn, then I would have no idea at all.
The net effect is that Google gets a random sample from billions of random conversations from millions of people every time this thing unintentionally goes off. They have a clear explanation as to why they got it and why ads are being served in response afterward. They can even make the case that the system is functioning as intended - after all, it'd be unreasonable to expect no false positives, or program bugs, or whatever, right? They can even say it's the user's fault and that they need to tune the voice model better.
Regardless, none of this changes the net result, which is they get a random sample of your conversation from time to time and are allowed to do whatever with it that they would have done if you sent it on purpose.
"Putting that together, I have no idea how many times the Google Assistant has activated in my pocket, gone live, recorded stuff, sent it to Google's servers, realized it wasn't a valid query, and shut off without alerting me."
Have you tried resisting an export from Google Takeout to see if there are answers in that data?
You may not be able to disassemble binaries or intercept network traffic but there are plenty of privacy researchers who can, and none of them have found anything.
That said, there's a much easier way to test this. Take two identical voice recognition smart devices (think Amazon Echos), register them each with a new never used Amazon account. Modify one of the devices to have a switch on its mic input which you leave off, and one which you leave on. See if the one with the mic on starts showing ads for things you've never searched for on that Amazon account. If the other one doesn't then there's your answer.
That sounds interesting enough that I might just give it a try.
Given the wide spread of this phenomenon and it’s been a decade, it’s either the most technically complex undetected global conspiracy or it’s not actually real[1].
2) If the app detects a consumption-related trigger word, the related conversation is flagged for transmission to the server.
3) Flagged audio block is converted to text. Consumption related verbs ("buy", "purchase", etc) are identified. The syntax of the sentence clearly indicates which noun is the target of a given consumption-related verb ("new car", "pool fencing")
4) Serve related ads