Apple's new Audio Intelligence turns the Watch into a listening device, privacy is part of the pitch

[Daniel Sims]

A hot potato: One of the main selling points of Apple's recently unveiled Watch Series 12 and Ultra 4 is a new Audio Intelligence feature that records and analyzes ambient speech, music, and other sounds. Seemingly anticipating the concerns that come with turning a smartwatch into a listening device, the company also outlined numerous privacy protections, but it remains to be seen how people will use Siri's new capabilities.

Pre-orders are now open for the Apple Watch Series 12 (starting at $399) and the Apple Watch Ultra 4 ($799), with availability starting on September 18. The new wearables introduce improved health-tracking features and can use Siri's upcoming AI enhancements to analyze and transcribe what they hear.

Apple's new Audio Intelligence includes four functions for the Watch Series 12 and Ultra 4, two of which deal directly with the user's conversations.

When the wearer double-presses the Digital Crown, Live Rewind displays a text snippet of what was said over the past 15 seconds, which users can save to the Siri app. Meanwhile, Siri Recap listens to conversations ambiently throughout the day to generate a summary later. Both features are set to enter beta in late 2026.

The other two features seem relatively benign. Sound Recognition alerts hard-of-hearing wearers to important noises such as sirens, alarms, doorbells, and crying children. Additionally, Music Recognition automatically uses Shazam to display a song's title and artist in the Smart Stack, without requiring users to tap the widget.

In a support article and a privacy document, Apple stresses that it processes audio in a secure part of the watch and iPhone that's inaccessible to the user and the company, so users can only retrieve text snippets or summaries, never the underlying audio. Live Rewind constantly overwrites old audio as new data streams in, and Siri Recap deletes audio after transcription.

Furthermore, Siri Recap condenses and encrypts transcripts to retain only a conversation's core meaning before sending them to Apple's Private Cloud Compute servers, where they remain inaccessible to the company, to create a summary.

Siri Recap automatically deletes summaries after seven days unless users save them. It also attempts to omit financial information, harmful content, government-issued IDs, and certain other identifying data.

Even if audio data and transcripts remain secure, predicting how people will use the new listening features will be difficult. Furthermore, deleting master audio data makes it harder to verify an AI summary's accuracy.

The Siri AI beta will roll out in English when watchOS 27 launches on September 14, with support for French, Japanese, Korean, Portuguese, and Spanish coming in October.

Compared with prior models, the Apple Watch Series 12 and Ultra 4 will also monitor heart rate and heart rate variability more accurately, and the Ultra 4 features improved battery life.

Permalink to story:

Apple's new Audio Intelligence turns the Watch into a listening device, privacy is part of the pitch

 

[psycros]

This is an intelligence agency's dream, and people will voluntarily put it on their wrists.

 

[ZedRM]

This is creepy tech.

 

[VitalyT]

So after a session on the toilet, you can ask Siri to turn it into music? I'm sold!

 

[LordVile95]

This is an intelligence agency's dream, and people will voluntarily put it on their wrists.

You carry you phone about and it’s not difficult for them to get into that, this seems less useful to tap into

 

[Anton Longshot]

I like and enjoy the Apple ecosystem (multiple devices) but I also consider some of their products totally canal-worthy (in-joke, I live in Amsterdam - people used to toss all their unwanted crap into the canals).

The usual ranty bit: devices can be nice/fun/lovely/woteffah buuuut the people who run Big Tech are nobody's friend. They're vultures and would do the unspeakable to get at your money if they weren't afraid of getting caught. And that's why they occasionally tiptoe the line between what's OK and what definitely isn't. Secretly hoping people will get used to what's NOT OK and in this manner render it axxeptable. They cannot be trusted and it's smart to keep an eye on what they're doing.

 

[acrossthestars]

You carry you phone about and it’s not difficult for them to get into that, this seems less useful to tap into

That would actually make this more intrusive, not less. It's "common knowledge", even if people often forget, that smartphones are ground zero for mass surveillance. We cry about Flock cameras and drone all of the time, by those devices are specifically "surveillance systems": that is LITERALLY their purpose. That smartphones can be used to surveil citizens is an unfortunate by product of what they are and how they interact with cellular infrastructure.

Meanwhile, Apple Watch is a smart watch. It's primary function is to tell time; that it make calls and monitors your body's homeostasis, in order to track things like blood-oxygen saturation and heart rate, is a by-product of the miscellaneous stuff Apple stapled onto the point of it being a timepiece. Everyone expects their smartphone to record everything about them, because that is how the smartphone privacy conversation is often framed.

Who is thinking to themselves, "oh yeah, my watch is ALSO a snitch..."?

 

[Anton Longshot]

Side note #4675 also a rant - the distracted generation (I.e. phone addicts) often assumes everyone's like them.
In my experience the opposite is true: many if not most people keep their phones in their pocket and furthermore don't use or even have social media apps. I rarely use mine outdoors except to take pics as I'm an avid and actually popular photographer.
Personally I find the addicts often annoying, especially if they stare at their phones when they shouldn't, and when they LOUDLY share their bluddy private life with everyone in a 6 meter radius making speakerphone calls.
It looks so sad to me: these people don't seem to have an actual life or hide from it as much as they can BECAUSE IT'S SO DIFFICULT. While that should be their problem it often becomes mine because of the things mentioned up there^^.
It's a strange world. Owning a phone is pretty much mandatory (I remember when mine bricked...hell I suddenly became a leper and companies literally refused to interact with me I swear. Id did not miss my cellphone when they didn't exist...but then I could say the same about PC's. Let me rephrase: it's often a stranger world than it seems. BIG SMILEY

 

[LordVile95]

That would actually make this more intrusive, not less. It's "common knowledge", even if people often forget, that smartphones are ground zero for mass surveillance. We cry about Flock cameras and drone all of the time, by those devices are specifically "surveillance systems": that is LITERALLY their purpose. That smartphones can be used to surveil citizens is an unfortunate by product of what they are and how they interact with cellular infrastructure.

Meanwhile, Apple Watch is a smart watch. It's primary function is to tell time; that it make calls and monitors your body's homeostasis, in order to track things like blood-oxygen saturation and heart rate, is a by-product of the miscellaneous stuff Apple stapled onto the point of it being a timepiece. Everyone expects their smartphone to record everything about them, because that is how the smartphone privacy conversation is often framed.

Who is thinking to themselves, "oh yeah, my watch is ALSO a snitch..."?

It’s literally harder to tap into from a hardware perspective by design and a software perspective.

 

[Megalomaniac]

Apple's new smartwatches can listen to your conversations and summarize them later...

In a court of law