Someone finally got an RTX 5090 running on a Mac - no hacks required

[zohaibahd]

In context: Back in 2018, Apple yanked Nvidia support from macOS entirely, and that was pretty much it for CUDA on the platform. Developers who wanted GPU compute from Team Green on their Macs were out of luck for years. But that's now changing.

Tiny Corp, the same company that built the tinybox AI accelerator, has written its own Nvidia GPU driver completely from scratch. It's called TinyGPU, and it's an open-source macOS kernel extension. Better yet, Apple has signed off on it. That means you don't need workarounds like setting up a virtual machine or messing with System Integrity Protection to run it. All you need to do is plug in an external GPU over Thunderbolt or USB4, approve the extension, and it works.

One YouTuber has already tested the driver using his RTX 5090 with 32GB of VRAM. Alex Ziskind plugged the GPU into a Mac Mini M4 Pro, and it ran just fine. That's impressive considering this is Nvidia Blackwell silicon hooked up to Apple Silicon via a single cable.

But actual performance is a more complicated story. In an inference test using Llama 3.1 8B – a popular open-source AI model – the RTX 5090 managed roughly 7.48 tokens per second. Tokens per second is basically how fast an AI model can spit out its response, and that number isn't going to blow anyone away.

Where things did get interesting, though, was in chat-style tasks. Time to first token, which is how long you wait before the response even starts appearing, was three to four times quicker than what you'd get from using Metal natively. That made the experience feel noticeably snappy, according to Ziskind.

Now for the catch. Running llama.cpp, which is one of the most popular open-source tools for local AI inference, on Metal is still about ten times faster overall. That's a big gap, though it's not really a Thunderbolt bandwidth problem. According to Ziskind, the real culprit is kernel efficiency: the 5090's memory can do 1.8TB/s, but the driver is only reaching 33GB/s.

Then again, this isn't meant to compete with llama.cpp right now. The point is that the driver, compiler pipeline, and memory management are all in place. With that out of the way, Tiny Corp can start working on optimizations.

Permalink to story:

Someone finally got an RTX 5090 running on a Mac – no hacks required

 

[Kashim]

Apple is so great. They only let you run software they control and you can only install hardware Apple allows. Or rather, you can't install any hardware at all. You can only have hardware that was already in there when you bought whatever garbage Apple sold you. That, or have Apple install whatever hardware they approve of, you know, for a nice premium. Then they wander why people hate Apple.

 

[CrisisDog]

But can it run Crysis?

 

[Endymio]

Apple is so great. They only let you run software they control and you can only install hardware Apple allows.

I'm not a huge fan of that approach, but the fact remains it adds stability, and a considerable segment of the population prefers it to an open ecosystem. That's the beauty of the free market system, vs. the ideals of economically-illiterate authoritarian types who continually demand the government "step in" and force companies into lockstep compliance, no matter how much it hurts consumers long term.

... Then they wander why people hate Apple.

I hate IG Farben for producing Zyklon B, despite knowing it was being used to commit genocide. I hate Technicatome for building enrichment-grade nuclear facilities for terrorist states. Hating Apple for giving consumers who want a walled-garden ecosystem is banal beyond words.

 

[Theinsanegamer]

But can it run Crysis?

Yes!

 

[Plutoisaplanet]

Apple is so great. They only let you run software they control and you can only install hardware Apple allows. Or rather, you can't install any hardware at all. You can only have hardware that was already in there when you bought whatever garbage Apple sold you. That, or have Apple install whatever hardware they approve of, you know, for a nice premium. Then they wander why people hate Apple.

Lol Apple literally approved the crappy third-party driver for RTX 5090 . So if you want to blame anyone for lack of support of the RTX 5090 on Macs, it would be Nvidia. The reason a lot of hardware isn't supported on Macs is because there's much less benefit than supporting Windows. Macs just have less market share than Windows.

 

[Theinsanegamer]

Lol Apple literally approved the crappy third-party driver for RTX 5090 . So if you want to blame anyone for lack of support of the RTX 5090 on Macs, it would be Nvidia. The reason a lot of hardware isn't supported on Macs is because there's much less benefit than supporting Windows. Macs just have less market share than Windows.

I have no idea how you dreamed that up. nVidia and AMD published drivers on MacOS right up until Apple themselves came out with the M1 and YEETed external GPU support to the curb. It was also apple that decided to not support newer APIs that the rest of the industry used, making support even more difficult.

Apple is absolutely deserving of blame here. Linux has a fraction the marketshare of MacOS for consumers yet they get up to date drivers without issue.

 

[Plutoisaplanet]

I have no idea how you dreamed that up. nVidia and AMD published drivers on MacOS right up until Apple themselves came out with the M1 and YEETed external GPU support to the curb. It was also apple that decided to not support newer APIs that the rest of the industry used, making support even more difficult.

Apple is absolutely deserving of blame here. Linux has a fraction the marketshare of MacOS for consumers yet they get up to date drivers without issue.

It says so in the article itself. The real problem is Nvidia can't make much money on Macs because Macs are only made/packaged by Apple. So why would Nvidia spend resources making a driver that's completely different for a product that would barely sell?

The reason Tiny Corp made an Nvidia driver was because they want developers to use their neural network framework, and they also want people to buy their computers for training. If you'd like, you can wire them $65k to get one, and that's exactly their play with a specialized driver here: https://tinygrad.org/