My first move into PC gaming was my uncle's Packard Bell Pentium II 233Mhz and I had to play Quake 2, Command and Conquer and Half Life in Software Mode. But back then, the graphics still looked considerably better than on the consoles - I believe Playstation and N64 were the biggest comparisons.
When I got my first PC: an HP 8470c Pavilion with Pentium III 450Mhz, 96MB SDRAM and a 20 GB HDD, it was around about the time Aliens vs. Predator was out and that game absolutely demanded a 3D accelerator card.
I wanted a Voodoo 3 3500 TV AGP because everything I had researched and experienced in the marketing led me to believe that that was the best card I could afford and it was better than most other options. I think I focused mostly on the amount of VRAM and the triangle counts.
I ended up buying the Voodoo 3 3000 instead because the price difference (due to the TV tuner) was so high and I was a lowly college student.
The Voodoo 3 3000 in OpenGL mode made Quake 2 look like a totally different game. Brighter lighting effects, darker, deeper textures. It was amazing. The difference in Half Life wasn't as dramatic, but I was able to run AvP with no issues in max settings. Unreal and Soldier of Fortune were beautiful as well.
As time went on, Voodoo made their way to the Voodoo 5 5500 but prices were so high, I ended up buying a Geforce MX400 instead which was a dramatic difference because I noticed way more detail in my flight sims like JANES USAF thanks to the increase in VRAM to 64MB.
I still have my Pavilion and if I were to build up a "classic" system to run classic games, I'd probably get the most powerful Voodoo I could run on it - even the 3500TV AGP - although the low Mhz would severely limit the number of games I could run to virtually everything prior to 2002. Many of my favorite games of that era will not run on my Windows 10, Core i9ex/2080Ti/64GB DDR4 simply because of the drivers. I would love to be able to play Yuri's Revenge and the rest of the C&C games but they won't run.
...or get the best Geforce card of the time.
And as you pointed out: Nostalgia will cost you...
That's the reason why I believe in spending the money to max out the RAM that the motherboard will handle and buying my accessories up front - including the best video cards of the time. This way, further down the road, the tower itself will be "as good as it can be" so if you choose to sell it to a nostalgia builder or ever return to it yourself, you won't have to worry about searching for better hardware because you'll already have the best you can have.
They want large amounts of cash for many of those classic cards now.
There's a Voodoo 5 5500 right now on Ebay being bidded up - but even if you bought it, you'd have a Hell of a time finding and installing drivers for it. The internet as is has requirements so high that many of the old systems won't even load Google now.