my first instinct was say delete 'em if you want to, you can always download again if every needed (and maybe an improved one too), but then.....
There might be a limited time span (a few years) after which they just disappear....
Then again, there are many sites such as DriverGuide.com which specialise in archiving old, old drivers.
So what do I do? I have an entire partition called 'setups' in which I keep the original downloaded .zip or .exe. I rename it to be sure exactly what it is though, most driver zips are totally cryptic.