Linux could have been brought down by backdoor found in widely used utility

[emorphy]

Why it matters: By happenstance Microsoft researcher Andres Freund found malicious code that could break sshd authentication. If it hadn't been discovered it could have posed a grave threat to Linux. The open source community has reacted to the incident, acknowledging the fortuitous nature of the discovery and how it was fortunately caught early before it could pose a significant risk to the broader Linux community.

Andres Freund, a PostgreSQL developer at Microsoft, was doing some routine micro-benchmarking when he noticed a small 600ms delay with ssh processes, noticing that these were using a surprising amount of CPU even though they should be failing immediately, according to his post on Mastodon.

One thing led to another and Freund eventually stumbled upon a supply-chain attack involving obfuscated malicious code in the XZ package. He posted his discovery on the Open Source Security Mailing List and the open source community took it from there.

I tried explaining my nontech friends today that an engineer debugging a 500ms delay has saved the entire web, potentially the entire civilisation

– Peer Richelsen – oss/acc (@peer_rich) March 30, 2024

The dev community has swiftly been uncovering how this attack was craftily injected into XZ utils, a small open-source project maintained by a single unpaid developer since at least 2009. The account associated with the offending commits seemingly played the long game, slowly gaining the trust of XZ's developer, which has led to speculation that the author of the malicious code is a sophisticated attacker, possibly affiliated with a nation-state agency.

Officially called CVE-2024-3094, it has the highest possible CVSS score of 10. Red Hat reports that the malicious code modifies functions within liblzma, which is a data compression library that is part of the XZ utils package and is a foundational part of several major Linux distributions.

Open source maintainer burnout is a clear and present security danger. What are we doing about that? https://t.co/GZETWimy5i

– Ian Coldwater �"��'� (@IanColdwater) March 29, 2024

This modified code can then be used by any software linked to the XZ library and allow for the interception and modification of data used with the library. Under certain conditions, according to Freund, this backdoor could allow a malicious actor to break sshd authentication, allowing the attacker to gain access to an affected system. Freund also reported that XZ utils versions 5.6.0 and 5.6.1 are impacted.

The xz backdoor is, well, setting a fire under the entire Linux ecosystem... but I'm also so impressed with how it was set up: 2-yr maintainership, oss-fuzz, etc.

...and who knows how long it would've stayed undetected if the injected sshd code ran faster (<600ms)

Highlights:

– Danny Lin (@kdrag0n) March 30, 2024

Red Hat has identified vulnerable packages in Fedora 41 and Fedora Rawhide, advising users to cease usage until an update is available, though Red Hat Enterprise Linux (RHEL) remains unaffected. SUSE has released updates for openSUSE (Tumbleweed or MicroOS). Debian Linux stable versions are safe, but testing, unstable, and experimental versions require xz-utils updates due to compromised packages. Kali Linux users who updated between March 26 and March 29 need to update again for a fix, while those who updated before March 26 are not impacted by this vulnerability.

However, as many security researchers have noted, the situation is still developing and more vulnerabilities could be discovered. It is also unclear what the payload was going to be. The US Cybersecurity and Infrastructure Security Agency has advised people to downgrade to an uncompromised XZ utils version, which would be earlier than 5.6.0. Security firms are also advising developers and users to conduct incident response tests to see if they've been impacted and if they have, to report it to CISA.

This is explains how the xz backdoor was found pic.twitter.com/n9rNjvawHU

– myq (@mippl3) March 30, 2024

Fortunately it doesn't appear as if those affected versions were incorporated into any production releases for major Linux distributions, but Will Dormann, a senior vulnerability analyst at security firm Analygence, told Ars Technica that this discovery was a close call. "Had it not been discovered, it would have been catastrophic to the world," he said.

Permalink to story.

https://www.techspot.com/news/102456-linux-could-have-brought-down-backdoor-found-widely.html

 

[Dimitriid]

It's a good thing the dev found it under Nadella and his 'Microsoft loves Linux' (Love might be perhaps too strong of a word but Nadella certainly enjoys the tremendous profits Linux has enabled all of his Azure business) because if he discovered this under say, Balmer era it would have probably been somehow exploited instead of getting reported.

...Well it perhaps has been already used and we just didn't know about because it was a high enough profile case that the powers that be didn't let anyone know how certain breaches happened.

 

[bviktor]

A classic example of manyeyeballs

 

[tragicallyhip]

A mysterious backdoor bodged together by a 'lone gunman' so sophisticated only a nation state could pull it off?, Torvalds was approached by an agency (Fill in blank) with an ask for a backdoor to Linux, told to piss up a rope they found another way to get what they wanted...almost

 

[trparky]

"The dev community has swiftly been uncovering how this attack was craftily injected into XZ utils, a small open-source project maintained by a single unpaid developer since at least 2009."

And yet this is a piece of software that nearly everything in Linux depends upon. How the hell is this guy not being compensated for his work? This is OpenSSL all over again.

 

[yannus]

Attacks on Linux just make it stronger. Sure that from now on many additional security procedures will be added.

 

[trparky]

Attacks on Linux just make it stronger. Sure that from now on many additional security procedures will be added.

Sure, until the next time the "thousand eyes" fail us yet again.

 

[gamerk2]

https://xkcd.com/2347/
Applies perfectly.

 

[Bruno M. Villar]

Attacks on Linux just make it stronger. Sure that from now on many additional security procedures will be added.

Or maybe turn it into a paid product for god's sake.

 

[trparky]

Or maybe turn it into a paid product for god's sake.

Exactly. People need to eat and that costs money. We don't live in Star Trek where one can just walk up to the wall and ask for a pizza and it materializes out of thin air.

 

[waclark]

Attacks on Linux just make it stronger. Sure that from now on many additional security procedures will be added.

You can only apply updates if you know where your vulnerabilities like. I won't attack "Linux" just because someone found a problem. Most all OSes these days have issues just waiting to be found and as new development happens on those OSes, more hacks will be found.

But I will say, it does prove the point that no OS is entirely safe and while Windows and MacOS have their warts, Linux is not free from attack vectors or bugs either.

 

[ZedRM]

It's a good thing the dev found it under Nadella and his 'Microsoft loves Linux' (Love might be perhaps too strong of a word but Nadella certainly enjoys the tremendous profits Linux has enabled all of his Azure business) because if he discovered this under say, Balmer era it would have probably been somehow exploited instead of getting reported.

...Well it perhaps has been already used and we just didn't know about because it was a high enough profile case that the powers that be didn't let anyone know how certain breaches happened.

Your opinion of Nadella is deeply flawed. He is no friend to the Linux community nor the consumer.

 

[ZedRM]

almost

Key word.. The scary part is that it is unlikely to be a US, UK or EU government entity.

 

[Watzupken]

Someone in MS found a critical flaw in Linux. But is Windows any safer? While they patch frequently, you can only patch what you know. And I will be happier if they patch and don't break any basic feature.

 

[yannus]

You can only apply updates if you know where your vulnerabilities like. I won't attack "Linux" just because someone found a problem. Most all OSes these days have issues just waiting to be found and as new development happens on those OSes, more hacks will be found.

But I will say, it does prove the point that no OS is entirely safe and while Windows and MacOS have their warts, Linux is not free from attack vectors or bugs either.

I'm referring to the way FOSS works. The transparency of its development makes that the whole community benefits from each mistake of every member.

And of course Linux, like any piece of software, has vulnerabilities. Nevertheless, there are MUCH less numerous and they are identifiable (as demonstrated in this story) and fixable by anyone with the competence to do so, something that needs "proprietary engineers" in Micro$oft or appl€ OSes.

 

[dangh]

This issue thankfully hasn't any widespread impact as only rolling distros were affected, and those are not meant to be used in anything productive.
And the advantage of this happening is putting additional eyes and resources to address this issue in the future, overall making linux more protected and reliable. That is the goal of foss.

 

[witMMXtechnology]

"Red Hat has identified vulnerable packages in Fedora 41"

Isn't Fedora on v39, with 40 just released as beta? what is Fedora 41? are there 'vulnerable packages' in Fedora 39 and 40?

 

[waclark]

I'm referring to the way FOSS works. The transparency of its development makes that the whole community benefits from each mistake of every member.

And of course Linux, like any piece of software, has vulnerabilities. Nevertheless, there are MUCH less numerous and they are identifiable (as demonstrated in this story) and fixable by anyone with the competence to do so, something that needs "proprietary engineers" in Micro$oft or appl€ OSes.

I think FOSS has pros and cons as evidenced by this article. Somone hijacked open-source code. For the most part it works pretty well but relying on "the community" to supply patches and updates isn't always a good option for some companies. That's why some of the larger distros have paid support like Red Hat, etc.

 

[anastrophe]

It's a good thing the dev found it under Nadella and his 'Microsoft loves Linux' (Love might be perhaps too strong of a word but Nadella certainly enjoys the tremendous profits Linux has enabled all of his Azure business) because if he discovered this under say, Balmer era it would have probably been somehow exploited instead of getting reported.

...Well it perhaps has been already used and we just didn't know about because it was a high enough profile case that the powers that be didn't let anyone know how certain breaches happened.

I've no more love for MS than anyone else, but this is reckless speculation. Do you have some evidence that Microsoft ever previously found a linux exploit and didn't report it? Found an exploit in any other operating system and didn't report it? I'm all ears.

It's no more responsible than me saying "I think Dimitriid probably found this vulnerability, exploited it, and didn't report it - what he wrote is the perfect distraction move to get attention off of him".

 

[anastrophe]

"The dev community has swiftly been uncovering how this attack was craftily injected into XZ utils, a small open-source project maintained by a single unpaid developer since at least 2009."

And yet this is a piece of software that nearly everything in Linux depends upon. How the hell is this guy not being compensated for his work? This is OpenSSL all over again.

'software that nearly everything in Linux depends upon'. No, not even slightly. It's important - perhaps extremely important in this specific context - but 'nearly everything' is a gross exaggeration.

 

[anastrophe]

Exactly. People need to eat and that costs money. We don't live in Star Trek where one can just walk up to the wall and ask for a pizza and it materializes out of thin air.

Linux can't be 'turned into' a paid product. That's a matter of fact and law. A distribution can be created and sold ("Redhat" e.g.), which primarily involves providing commercial/enterprise _support_ for the product and operation of it, but the most widely used distributions are free and can't be "converted" into a proprietary closed format for sale.

 

[trparky]

Linux can't be 'turned into' a paid product. That's a matter of fact and law. A distribution can be created and sold ("Redhat" e.g.), which primarily involves providing commercial/enterprise _support_ for the product and operation of it, but the most widely used distributions are free and can't be "converted" into a proprietary closed format for sale.

And that's why eventually I see the death of Linux and why Github is littered with dead projects. As for Linux, there's a major shortage of people who are willing to step up to be kernel maintainers and yes, Linus has been quoted as saying that. When Linus dies, who'll pick up the torch?

People who have the expertise and talent to do the low-level kernel work usually want to be paid for their work and rightfully so.

 

[anastrophe]

And that's why eventually I see the death of Linux and why Github is littered with dead projects. As for Linux, there's a major shortage of people who are willing to step up to be kernel maintainers and yes, Linus has been quoted as saying that. When Linus dies, who'll pick up the torch?

People who have the expertise and talent to do the low-level kernel work usually want to be paid for their work and rightfully so.

96% of the top one million websites run on linux. Most of the rest of the internet's infrastructure globally runs on linux (routers aside).
Most contributors of the core portions of linux are otherwise employed, often employed specifically by companies to perform their work on linux.
Torvald's quote on the shortage is from four years ago, a lifetime in computer ecosystems. Since he hasn't brought it up since then (to my knowledge), perhaps there isn't any more.

linux isn't dying any time soon, too much of the world is completely reliant upon it.

 

[ZedRM]

And that's why eventually I see the death of Linux

This will never happen. Linux is WAY too ubiquitous to disappear anytime soon. Android is the most used OS on Earth and is based very closely on the Linux kernel. Many have argued, including me, that Android is a Linux distro. You can't easily kill something the whole world uses.

 

[trparky]

This will never happen. Linux is WAY too ubiquitous to disappear anytime soon. Android is the most used OS on Earth and is based very closely on the Linux kernel. Many have argued, including me, that Android is a Linux distro. You can't easily kill something the whole world uses.

Fine, but I see the kernel being forked for Android and eventually becoming so drastically changed that you'll barely know that it was based upon the Linux kernel.

It's just like how the kernel of MacOS is based upon the BSD kernel but you'd hardly know it today.