Windows 11 preview references AI feature that searches inside audio and video files for specific words

[midian182]

A hot potato: It's easy to imagine that after the Recall controversy, Microsoft would tread carefully when it comes to AI-powered features that rifle through the contents of users' PCs. However, references to an 'intelligent media search' in a Windows 11 Canary build suggest lessons haven't been learned: the function can search for specific spoken words in your stored audio and video files.

Intelligent media search, referenced in Canary Channel build 27695 (spotted by @XenoPanther), works by transcribing all audio files and videos saved on a Windows PC so the spoken words are searchable.

These changes may already exist in other branches but in 27695...

"Intelligent media search" references have been added
Intelligent media search available
Search by spoken words in your indexed video or audio files. By clicking 'I agree,' you consent to scanning the media files…

– Xeno (@XenoPanther) August 30, 2024

The idea behind the tool is that users could, for example, find something that was mentioned in a recorded meeting, or for YouTube video creators to search through their extensive libraries for a certain moment. It could also be used to organize files that mention a particular topic.

Microsoft's description asks users to consent to their media files on a device being scanned. An AI model will be downloaded and installed, which transcribes the user's local media files and indexes them before enabling content-based search.

It does seem surprising that Microsoft is going down this path once again after the Recall controversy. Announced on May 20 and initially set for an official launch in the Copilot+ PCs, the feature constantly takes screenshots of whatever you're doing every few seconds, storing the snapshots so users can search through a scrollable timeline to find something they've been working on.

Despite Microsoft announcing that Recall would be opt-in and the search index database encrypted, the outcry against the feature's privacy implications continued. The Redmond firm eventually said it was delaying Recall's launch. It will return in Insider builds this October.

It's likely that, as with Recall, Intelligent media search will be a feature exclusive to the Copilot+ PCs that arrived in June. Microsoft requires these devices to offer an NPU with at least 40 trillion operations per second (TOPS).

Unsurprisingly, plenty of people (though not everyone) have spoken out against intelligent media search, noting that Microsoft could use the information it scans on a person's PC to further train its AI models or even for targeted advertising purposes. There are also the inherent security risks associated with such a feature.

Permalink to story:

Windows 11 preview references AI feature that searches inside audio and video files for specific words

 

[ScottSoapbox]

This isn’t as bad if the opt in is very clear. I already have Windows index my Word and code files for easy searching for work files.

Videos would depend on what you keep on your computer. Still not as terrible as tracking everything you do ever second of every day.

 

[Puiu]

As long as it is local indexing that's fine.

 

[dakks]

I have been waiting for something like this for a very long time. I have, in the past, spent hours searching for a specific video file. This helps me tremulously. Please turn down the volume on the paranoia.

 

[Theinsanegamer]

"It does seem surprising that Microsoft is going down this path once again after the Recall controversy"

Why is this surprising to you? This is Microsoft, who has long stopped caring about the consumer and does whatever it wants, damn the consequences. They do not care about you, you are a walking wallet to them, and your data WILL be scraped and sold to the government and anyone who wants it. Delaying Recall was only to let the controversy die down, they didnt actually change anything, and they dont care if your data gets stolen or used against you.

Wake up, FFS.

 

[dakks]

"It does seem surprising that Microsoft is going down this path once again after the Recall controversy"

Why is this surprising to you? This is Microsoft, who has long stopped caring about the consumer and does whatever it wants, damn the consequences. They do not care about you, you are a walking wallet to them, and your data WILL be scraped and sold to the government and anyone who wants it. Delaying Recall was only to let the controversy die down, they didnt actually change anything, and they dont care if your data gets stolen or used against you.


I have approximately 18tb of video files. I need a search function. You want to go back to horse & buggy, be my guest, but don't you dare drag the rest of us w/you. & btw, EVERY American corporation for the last decade or so, has been abusing us & unnecessarily draining our wallets & then kicking us to the curb.How do you not know that? Homeschooled, living in a cave. What!

 

[Watzupken]

I don’t think this is surprising. As technology advances, Microsoft has increasingly added more ways to inject telemetry and monitoring features into the OS. They called them Windows features, but they are nothing more than Spyware and Trojans that are deemed “safe” because Microsoft is the owner. Ultimately, I feel users have options such as Linux to power their PCs. It may not be similar to Windows, but it’s a matter of getting used to it. I’ve had enough of Microsoft’s shady practices and moved mostly to using Linux since the start of this year. Sure I missed some Windows conveniences, but I feel it’s a feature rich and light OS, and mostly without big tech forcing unwanted change on us. Microsoft don’t even let us uninstall features that we don’t use and that to me don’t feel right, ie, the app may still be doing something in the background even if the user disables it.

 

[ZedRM]

Personally, I will not tolerate an AI running on my PC. Microsoft can cram it's CoPilot up it's bum..

 

[OortCloud]

This isn’t as bad if the opt in is very clear. I already have Windows index my Word and code files for easy searching for work files.
Videos would depend on what you keep on your computer. Still not as terrible as tracking everything you do ever second of every day.

I agree, but lets hope they don't make this on by default at some point and keep 'accidentally' switching off the option to share this data with Microsoft.
Also, I suspect this is the thin end of the Recall-wedge being gently re-inserted! That sounds very much like exactly the same model which would be used to analyse those screenshots Recall wants to take.