Siri's new AI smarts fail at sports trivia, claims Philadelphia Eagles won 33 Super Bowls

[zohaibahd]

Facepalm: Apple's much-hyped Siri integration with ChatGPT may have added a ton of useful functionality, but it's apparently done little to improve the digital assistant's knowledge of sports. A damning report highlights just how abysmally Siri performs at recalling simple facts like past Super Bowl winners.

According to the report from One Foot Tsunami's Paul Kafasis, when asked "Who won Super Bowl?" for each number from 1 through 60, Siri correctly provided the winner for only 20 out of the 58 Super Bowls that have been played so far. That translates to a success rate of just 34%.

The details get even more embarrassing. Kafasis found that at its worst, Siri missed an incredible 15 Super Bowl winners in a row from Super Bowl XVII through XXXII. And in a baffling mishap, it erroneously credited the Philadelphia Eagles with a whopping 33 non-existent Super Bowl wins.

Kafasis documented every single one of Siri's wrong answers in a downloadable spreadsheet, which you can find on his blog.

Another report by Daring Fireball's John Gruber corroborated these findings. It found that AI assistants and search engines from Google, Anthropic, and DuckDuckGo fared far better when asked similar Super Bowl trivia questions. Gruber even found they could all handle not just past results, but smartly dodge trick questions about future Super Bowls that haven't happened yet.

Siri's poor performance doesn't stop at sports trivia either. Daring Fireball posed a more obscure question – "Who won the 2004 North Dakota high school boys' state basketball championship?" Incredibly, both Kagi and ChatGPT provided the fully correct answer, with the latter earning bonus points for including a link to video of the championship game. Meanwhile, Siri once again got it wrong.

What makes all this particularly damning is that Siri's essentially powered by the same ChatGPT that fares perfectly fine when used without Siri.

The blog also points out that the old pre-AI versions of Siri at least acknowledged limitations on such queries and provided relevant web links. But new Siri powered by Apple Intelligence with ChatGPT integration lies with confidence – a hallmark of unrefined AI – making it worse than its predecessor.

Both blogs conclude that Apple has more work to do in this area. As Gruber bluntly states, Siri with ChatGPT is currently "a massive regression" over the old Siri when it comes to handling simple factual prompts.

Permalink to story:

Siri's new AI smarts fail at sports trivia, claims Philadelphia Eagles won 33 Super Bowls

 

[pnntmp]

So is this a Siri or a ChatGPT failure?
Which one is giving the wrong answers? If Siri is now just a wrapper around ChatGPT, it's ChatGPT to blame, and Siri .. kind of doesn't exist anymore.

Can we say Siri is possessed by ChatGPT?

 

[The Talking Tech]

Siri???
That's so 2020

 

[Squid Surprise]

So is this a Siri or a ChatGPT failure?
Which one is giving the wrong answers? If Siri is now just a wrapper around ChatGPT, it's ChatGPT to blame, and Siri .. kind of doesn't exist anymore.

Can we say Siri is possessed by ChatGPT?

If you read the article, you’d know it was Siri - as ChatGPT on its own got the correct answers.

 

[Burty117]

I must be one of the few people that's really enjoyed Apple Intelligence so far.

The notifications truly make me laugh, some of the stuff it summarises is just hilarious, whenever I ask Siri something, I know I'm getting the wrong answer or no answer at all (unless its something simple like the weather or to turn the lights off).

Maybe because I went in with the expectation none of it was going to work very well, I've just found it entertaining? Whilst a lot of people went in thinking it would be great and are sorely dissapointed?

 

[MakeMSGreatAgain]

Siri seems to get worse with each new iOS version. I mean some features like being able to just say Siri instead of Hey Siri and having it still listening after finishing a command without you having to say "Siri" again is helpful. But it feels to me the actual effectiveness and understanding by Siri has gone down.

 

[BabyFaceLee]

There is absolutely no point in using AI if it can provide inaccurate information (e.g. erroneous iOS notification summaries are a case in point). The rush to implement AI at this stage is potentially dangerous. Nuance dear boy, nuance. AI should really be amended to AE....Acquiring Experience...it's got a long way to go.

 

[Squid Surprise]

There is absolutely no point in using AI if it can provide inaccurate information (e.g. erroneous iOS notification summaries are a case in point). The rush to implement AI at this stage is potentially dangerous. Nuance dear boy, nuance. AI should really be amended to AE....Acquiring Experience...it's got a long way to go.

While I agree that rushing to use AI might not be a good idea… not using something because it MIGHT give inaccurate information would mean we should stop using the internet…

 

[captaincranky]

I'm from Philly. Believe me, there were times when the Eagles could go several seasons without winning 33 GAMES.

 

[BabyFaceLee]

While I agree that rushing to use AI might not be a good idea… not using something because it MIGHT give inaccurate information would mean we should stop using the internet…

That's a valid point but in this case the whole point of using AI is to obtain valuable information from existing sources. The information on the internet is certainly sometimes inaccurate or false because we humans have deliberately or otherwise posted it. AI should not do the same I.e. make stuff up! I can't imagine there is a website that proclaims that the Philadelphia Eagles have won the Superbowl 33 times so what on earth is it doing? That's concerning.