Most AIs struggle with reading clocks, misreading faces 75% of the time

[midian182]

Facepalm: Generative AI tools are able to perform the sorts of tasks that once seemed the stuff of sci-fi, but most of them still struggle with many basic skills, including reading analog clocks and calendars. A new study has found that overall, AI systems read clock faces correctly less than a quarter of the time.

A team of researchers at Edinburgh University tested some top multimodal large language models to see how well they could answer questions based on images of clocks and calendars.

The systems being tested were Google DeepMind's Gemini 2.0, Anthropic's Claude 3.5 Sonnet, Meta's Llama 3.2-11B-Vision-Instruct, Alibaba's Qwen2-VL7B-Instruct, ModelBest's MiniCPM-V-2.6, and OpenAI's GPT-4o and GPT-o1.

Various types of clocks appeared in the images: some with Roman numerals, those with and without seconds hands, different colored dials, etc.

The systems read the clocks correctly less than 25% of the time. They struggled more with clocks that used Roman numerals and stylized hands.

The AI's performance didn't improve when the seconds hand was removed, leading researchers to suggest that the problem comes from detecting the clocks' hands and interpreting the angles on a clock face.

Using 10 years of calendar images, the researchers asked questions such as what day of the week is New Year's Day? and What is the 153rd day of the year?

Even the most successful AI models got the calendar questions wrong 20 percent of the time.

The success rates varied based on the AI system being used. Gemini-2.0 was the highest scorer in the clock test, while GPT-01 was accurate 80% of the time on the calendar questions.

"Most people can tell the time and use calendars from an early age," said study lead Rohit Saxena, from Edinburgh University's School of Informatics. "Our findings highlight a significant gap in the ability of AI to carry out what are quite basic skills for people. These shortfalls must be addressed if AI systems are to be successfully integrated into time-sensitive, real-world applications, such as scheduling, automation and assistive technologies."

Aryo Gema, another researcher from Edinburgh's School of Informatics, said, "AI research today often emphasises complex reasoning tasks, but ironically, many systems still struggle when it comes to simpler, everyday tasks."

The findings are being reported in a peer-reviewed paper that will be presented at the Reasoning and Planning for Large Language Models workshop at The Thirteenth International Conference on Learning Representations (ICLR) in Singapore on April 28. The findings are currently available on the preprint server arXiv.

This isn't the first study this month showing AI systems still make plenty of mistakes. The Tow Center for Digital Journalism studied eight AI search engines and found that they are inaccurate 60 percent of the time. The worst culprit was Grok-3, which was 94 percent inaccurate.

Permalink to story:

Most AIs struggle with reading clocks, misreading faces 75% of the time

 

[scoffer]

If these AI gits get away scot-free with scraping places like Z-Library et al....don't come knocking on MY f@@@#$g door when I do the same!!!

 

[ScottSoapbox]

This is especially funny given the plan to sell us AI assistants that will book us dinner reservations and other things requiring a basic understanding of calendars and time.

 

[Str8Shot]

Believe it or not, but a few years ago, I read a report back when AI was still in its infancy, where it was described that cursive handwriting was impossible for AI to decipher, and at the time it was suggested that there was a steady but deliberate attempt by the educational system to move away from cursive so that when AI would mature, it too would be able to decipher everything going forward (so that nothing would slip under the radar).

Like Elon Musk said: “with AI we are summoning the demon.” People better wake up before this tech that no one asked for, controls the whole world via AI Governance and IoT/IoB, because that’s where this is headed, thanks in part from our “hero” politicians that promise to save us all.

 

[Thatsdisgusting]

Misreading faces 75% of the time

THANK GOD! I wish it was 95% - ok-leave those 5% to those pedos that need to be found

 

[ScottSoapbox]

Believe it or not, but a few years ago, I read a report back when AI was still in its infancy, where it was described that cursive handwriting was impossible for AI to decipher, and at the time it was suggested that there was a steady but deliberate attempt by the educational system to move away from cursive so that when AI would mature, it too would be able to decipher everything going forward (so that nothing would slip under the radar).

When have you ever used cursive outside of signing your name?

Replacing it with typing would be a more useful skill (although that may soon be replaced by speech-to-text for most people).

 

[RudyBob]

AI can't read a clock? There is a large population of students that can't read clocks

 

[umeng2002]

What's the point of using AI like this if you have to double check everything they say?

 

[New Day]

I remember having an issue developing reading a handed clock in early early years, having to remember the minutes related to each number, etc, so I can imagine the kind of issue these systems have. (Plus those clocks in the above image look pretty, and pretty goddamn distracting. EWw.) Most of the issues I've seen reported have made immediate sense to me. They should come talk with me.....

When have you ever used cursive outside of signing your name?

Replacing it with typing would be a more useful skill (although that may soon be replaced by speech-to-text for most people).

Before the 70s, people printed. Then perhaps through some sociological phenomenon that bled into academia/pedagogy, cursive got heavily promoted. I hated printing, as it took longer, and mine I thought looked like *** - unlike my parents'. I rarely have physically written in adulthood, particularly in the last two and some decades, typing class in high school for the win, but intentionally improved my print writing to a modicum of satisfaction.

 

[Tantor]

I remember having an issue developing reading a handed clock in early early years, having to remember the minutes related to each number, etc, so I can imagine the kind of issue these systems have. (Plus those clocks in the above image look pretty, and pretty goddamn distracting. EWw.) Most of the issues I've seen reported have made immediate sense to me. They should come talk with me.....




Before the 70s, people printed. Then perhaps through some sociological phenomenon that bled into academia/pedagogy, cursive got heavily promoted. I hated printing, as it took longer, and mine I thought looked like *** - unlike my parents'. I rarely have physically written in adulthood, particularly in the last two and some decades, typing class in high school for the win, but intentionally improved my print writing to a modicum of satisfaction.

I believe that handwriting is incorrectly taught. My handwriting looked like garbage until I finally grasped the importance of strong horizontal and vertical strokes. Instead of curves, which are difficult.

Cursive was extensively taught in the 1800's. My great grandmother was born in 1870, and my grandmother in 1901. They both had gorgeous cursive handwriting.

I grew up in the 1950's and 60's. Cursive was a requirement.

 

[kiwigraeme]

LLM not being able to add as well. think it depends on how its asked. maybe wrong answers are just when 2 numbers show up together with a wrong answer somewhere in the training data
Given all the kids books , and maths book telling us how to do it . Now Jemina the little hand is pointing to the 2 and the big hand
Same for addition add that with that and if greater than 9 carry the one etc
just tested basis chatgpt added just find , probably now uses a calculator- why shouldn't it

Do LLM have visualisation or special tools - clock made be hard to read - but transpose data to easy visualised clock
Humans do a mind visualisation ( well some people can't ) one of reasons for illusions we fll it stuff we think is there , or change colours to making real world lighting sense

asked chatgpt if use calculator said no , ask how :-

1. Start by adding the numbers column by column, starting from the rightmost digits:

markdown
Copy
2546
+ 7848
-------

1. Add the digits in the ones place: 6 + 8 = 14. Write down 4, and carry over the 1.
2. Add the digits in the tens place: 4 + 4 = 8, plus the carry-over 1, making it 9.
3. Add the digits in the hundreds place: 5 + 8 = 13. Write down 3, and carry over the 1.
4. Add the digits in the thousands place: 2 + 7 = 9, plus the carry-over 1, making it 10.

So, the result is 10,394!


This process is just a basic breakdown of how we do column addition step by step.


asked basic Chatgpt (not logged in )

on a clock if big hand is between the 7 and 8 and small hand is on the 2 what time is it?

ChatGPT said:​

If the big hand (minute hand) is between the 7 and 8, that means the time is somewhere between the 35th and 40th minute. The small hand (hour hand) on the 2 means it's in the 2 o'clock hour.
So, the time would be approximately 2:37.

 

[New Day]

I believe that handwriting is incorrectly taught. My handwriting looked like garbage until I finally grasped the importance of strong horizontal and vertical strokes. Instead of curves, which are difficult.

Cursive was extensively taught in the 1800's. My great grandmother was born in 1870, and my grandmother in 1901. They both had gorgeous cursive handwriting.

I grew up in the 1950's and 60's. Cursive was a requirement.

Yes, you would know better than I, who saw and remembers seeing mainly print. As for lines and strokes, that brings up memories of the teacher saying that and all. I drew just as terrible as I printed. And what hasn't come easy I have tended to pass on. I detest difficulty.

 

[Tantor]

Yes, you would know better than I, who saw and remembers seeing mainly print. As for lines and strokes, that brings up memories of the teacher saying that and all. I drew just as terrible as I printed. And what hasn't come easy I have tended to pass on. I detest difficulty.

I agree with detesting difficulty.

All forms of writing are very difficult to master. Printing, calligraphy, cursive, Chinese pictograms. But once you're over the hump, it becomes a source of great satisfaction.

Cursive is a European art form.

 

[New Day]

I agree with detesting difficulty.

All forms of writing are very difficult to master. Printing, calligraphy, cursive, Chinese pictograms. But once you're over the hump, it becomes a source of great satisfaction.

Cursive is a European art form.

I admire those things as I admire Form (which is plural-er, form or forms?....my music instructor asked this early on in his non-major theory class), but I've gotten no satisfaction out of things that gave me grief. I've wanted to do something until I haven't, and then have left it.

Indeed, none of us nor most of the world's population would have birthed within such a sensibility.....and yet, nor would we have all the felgarcarb.