Google takes the gold in annual smart speaker IQ test

Shawn Knight

Posts: 14,458   +171
Staff member
Why it matters: Smart speakers are handy for quick queries without reaching for your smartphone or firing up your computer but as the most recent test data shows, their ability to return a correct answer needs work in most cases.

Digital assistants and the smart speakers that bring them into our homes have exploded in popularity in recent years as consumers exhibit little concern regarding privacy. With so many options now available to choose from, accuracy is just as important a metric to consider than brand loyalty, aesthetics or other characteristics when selecting a digital assistant.

Loup Ventures recently tested four leading smart speakers – Apple HomePod, Amazon Echo, Google Home and Invoke – by asking Siri, Alexa, Google Assistant and Cortana a series of 800 questions. According to their results, Google’s solution came out on top as it understood 100 percent of queries and answered 87.9 percent of them correctly.

Apple’s Siri finished in second, understanding 99.6 percent of questions asked and getting 74.6 percent of them correct. That was narrowly enough to edge out Alexa, which understood 99.0 percent of questions but only got 72.5 percent of them right. In last place was Cortana. Although it had a higher level of understanding than Alexa at 99.4 percent, it only managed to answer questions correctly 63.4 percent of the time.

It’s worth noting that Loup Ventures separates digital assistants on your smartphone from those on speakers because the use cases and user experiences greatly differ.

Lead image courtesy Daisy Daisy via Shutterstock

Permalink to story.

 

Axiarus

Posts: 627   +422
It's good to know what spy system for my home is the most useful.
I use an Android phone. I am already being watched by Google. Recently invested in Google Home, smart bulbs/sockets and I love it. Have not seen any targeted ads from conversations yet.
 

onestepforward

Posts: 165   +73
Bahaha, I needed a good laugh tonight.
Yes, we all need a good laugh but laughing provides no evidence of the performance of Bixby. Bixby is regularly criticised for its poor performance, but this is by way of anecdote. Testing Bixby along with the others would have provided better evidence of its performance.
 

captaincranky

Posts: 19,162   +8,307
These smart speakers seem to have a reciprocal effect on the intelligence of their owners.

As the little black boxes get smarter, the people who to let them in seem to get dumber.

It's sort of like having to pay "Big Brother", to put a "Telescreen" in your home.
 

Hexic

Posts: 1,265   +1,986
TechSpot Elite
Yes, we all need a good laugh but laughing provides no evidence of the performance of Bixby. Bixby is regularly criticised for its poor performance, but this is by way of anecdote. Testing Bixby along with the others would have provided better evidence of its performance.

Bixby's performance is underwhelming to the point of being a non-essential metric. Until Bixby dramatically improves, it is a moot point including it in any metrics with Google Assistant & friends.
 

onestepforward

Posts: 165   +73
Bixby's performance is underwhelming to the point of being a non-essential metric. Until Bixby dramatically improves, it is a moot point including it in any metrics with Google Assistant & friends.

That is your opinion. It might be correct, but without being tested, it remains nothing more than opinion.