CERN is moving more than 2,200 specialized computers from RHEL to Debian

Alfonso Maruccia

Posts: 2,704   +1,029
Staff
Looking ahead: CERN engineers have unveiled a complex plan to move a number of computers to a new Linux distribution. The upgrade is required to prevent older, specialized hardware from becoming obsolete, though it's turning out to be a challenging endeavor.

During a recent conference held in Winterthur, Switzerland, engineers working at the European Organization for Nuclear Research (CERN) confirmed that the laboratory is going to move from Red Hat Enterprise Linux to Debian. The switch will affect more than 2,200 computers, many of which cannot be upgraded to newer RHEL releases because newer Red Hat releases will leave much older hardware behind.

As one of the world's leading research institutions in particle physics, CERN generates massive amounts of data on a regular basis. The lab where Tim Berners-Lee created the World Wide Web's original vision was already packing 49 petabytes of digital files back in 2016. All this data requires a solid, ever-growing IT infrastructure, and the same goes for machines designed to manage scientific experiments through custom hardware boards.

CERN was an early adopter of the RHEL ecosystem. Over the past few years, the institute has been using the commercial system's free variant AlmaLinux. The lab initially considered switching to CentOS Stream, which is one of the upstream sources for RHEL development. The engineers ultimately decided to abandon the entire RHEL infrastructure after the CentOS project decided to implement a disruptive change in the system's compiler.

CentOS Stream 9, and thus RHEL 9, are built with the "-march=x86-64-v2" compiler flag. The x86-64v2 revision of the x86 ISA includes a few specific architectural changes, breaking compatibility with 64-bit CPUs released before 2009.Lack of support for pre-x86-64-v2 CPUs would force CERN to replace 47% of its embedded control computers. Even worse, since 2025, RHEL 10 has moved to x86-64-v3, which means an additional 17% of CERN's computers would need to go to a landfill sooner rather than later.

CERN engineers explained that Debian 13 might save the institute a massive IT replacement job and related costs. The free, "universal" operating system is the second-oldest Linux distribution still in development behind Slackware, and serves as the basis for numerous popular distributions, most notably Ubuntu.

Still, replacing RHEL/AlmaLinux with Debian is proving to be a challenge of its own. CERN uses many custom hardware solutions, which require specific kernel drivers, bootloaders, and system images.

Debian does not provide a standard toolset for automating package building and publishing packages, which means the engineers are going to evaluate some additional provisioning solutions such as Debusine. Nevertheless, it's better than throwing away thousands of perfectly functional computers and spending millions to build an entire IT infrastructure from scratch.

Permalink to story:

 
Tbh CERN kinda did this to themselves. Starting in 2004, CERN in conjunction with Fermilab and other universities helped develop and maintain a fork of RHEL called Scientific Linux. They supported custom software and hardware as part of this project; that was the point of it.

In 2015, CERN was the first institute to bail on Scientific Linux and rely on CentOS/RHEL. Within a few short years, Fermilab struggled to maintain Scientific Linux on their own and also switched to CentOS. Now beholden to Red Hat's target market, CERN finds themselves in this situation.

CERN is probably better off cost-wise relying on Debian going forward, but planned obsolescence was a risk they decided to accept. The thousands of computers they struggled to find support for was a result of their own decisions. Besides, RHEL 9 was released in 2022 and there were reports almost 2 years earlier that it wouldn't support x86-64-v1: https://www.phoronix.com/news/RHEL9-Drop-Old-CPUs-Fedora-ELN

It's not surprising things came to this. Working in higher education tech myself, most universities decided to outsource almost everything and become beholden to that loss of control. That means growing costs, loss of custom features, and risk of service failure by third parties: https://en.wikipedia.org/wiki/2026_Canvas_data_breach
 
Last edited:
I know it's RH's choice to make these changes but jeepers it does seem over the top. Reminds me of M$'s similar actions with enforcing certain Win11 minimums. And unsurprisingly, is now offering to upgrade to Win11 on much older hardware again. ie: The minimums were always arbitrary for non-technical reasons.
 
I consider it among my top two, standing tall with Evangelion. How it's resolved towards the end, particularly episode 23, the turning point, is simply mind blowing :- )
If you want your mind truly boggled, nothing beats Excel Saga.
Especially the last episode titled "going too far" which does as it says on the tin. Deliberately made as not to be broadcast on TV.
(That episode in particular also has a great outro)
 
If you want your mind truly boggled, nothing beats Excel Saga.
Especially the last episode titled "going too far" which does as it says on the tin. Deliberately made as not to be broadcast on TV.
(That episode in particular also has a great outro)
Thanks; I'll have to take a look, especially the last episode, since Wikipedia it says it was deemed too extreme for TV.

SG was never extreme in that fashion, but what they achieved in the plot was nothing short of astonishing, and brilliantly, they did it again with SG 0.
 
Back