Your smart speaker is always listening
Are you ever really alone when talking to your smart speaker? It turns out there might just be someone on the other side listening to you, especially when your voice commands go haywire. That’s according to Anthonio Pettit, a writer for Microsoft who’s worked on voice platforms like Microsoft’s Cortana, Samsung’s Bixby and Amazon’s Echo. He was even part of a team responsible for listening in on the times when you might have teased your Echo. (“Alexa, get me a beer,” is something he’s heard a lot.)
Pettit was part of a recent episode of KUOW’s “Prime(d)” podcast on smart speaker technology, including the “quality assurance” bin where accidental and pear-shaped voice commands wind up. “Prime(d)” co-host Joshua McNichols talked with Marketplace Tech host Molly Wood about his reporting on the range of audio Amazon collects. And if you were wondering, it’s pretty broad. The following is an edited transcript of their conversation.
Joshua McNichols: Those sort of accidental moments where maybe a cat knocked over a vase and it made a sound that sort of sounded like “Alexa” when the vase fell — those recordings go into a bucket called quality assurance, where somebody is listening to them and classifying “Why did this not turn out as we had planned?” We’ve written all these pairs of questions and answers like, “If humans say this, then Alexa should return this answer,” but some of those just aren’t classifiable, and they have to judge what went wrong.
Molly Wood: How did they, before we dig into the implications of that, how do they do that? I mean, they really just apply human logic to it? [They] listen to it and think “Oh, OK, this doesn’t match the script”? Or is there also some engineering that goes into it?
McNichols: Exactly. Well, I’m sure there’s algorithms that help them with some of them, but a lot of them go to a human, and a human listens to them because humans are still better at interpreting human speech in many ways than artificial intelligence.
Wood: How hard would it be for one of these engineers, one of these people in the quality assurance department, to figure out who the user is?
McNichols: Well, the information that comes to them is anonymized. But who knows what might be recorded in that time period? It could be some information that could be identifiable. And if somebody managed to attack one of these companies and get these recordings or something, then that’s another thing. But we don’t have any evidence that that has happened yet.
Wood: Well, because even anonymized, there have been plenty of incidents throughout technology’s history of anonymized data being pretty easily tracked back to an IP address.
McNichols: Yeah, that’s right. And we know that data that may not be easily ascribed to you at this time at some point in the future may be more easily correlated with you by combining different data sets that creates associations that aren’t in one set of data alone.
Related links: more insight from Molly Wood
Elsewhere in Amazon news: CEO Jeff Bezos put out a letter to shareholders this week in which he talked about the company’s failures and said that the development of the Echo smart speaker started around the same time as the company’s biggest financial failure — the Amazon Fire phone. Yeah, remember that?
It made me think, because at the time it seemed really obvious that Amazon might need a phone to complete its ecosystem since most of our shopping, browsing and ad delivery was becoming mobile. But instead, smart speakers are among the fastest growing categories in consumer electronics.
On Thursday, Credit Suisse assigned a neutral rating to Apple and said it expected the company’s smartphone sales to decline by double digits because, well, phones have peaked.
Bezos’ letter said the company will have to keep experimenting with potentially multibillion-dollar failures in order to keep innovating.
One other note on Amazon.
The company has been criticized for how it treats its workers at fulfillment centers and warehouses: low pay, long hours and really not fun working conditions — including people peeing in bottles because they’re afraid to leave their desks.
Well, on Thursday, pilots at Amazon Air protested over low pay and what they said are terrible working conditions. They also feel the pressure to keep supply chain costs low for Amazon, creating bad conditions for cargo pilots across that industry.
That’s maybe not the ideal way to finance the multibillion-dollar experiments.
There’s a lot happening in the world. Through it all, Marketplace is here for you.
You rely on Marketplace to break down the world’s events and tell you how it affects you in a fact-based, approachable way. We rely on your financial support to keep making that possible.
Your donation today powers the independent journalism that you rely on. For just $5/month, you can help sustain Marketplace so we can keep reporting on the things that matter to you.
