No, I think the speed matters. It's the slowest part and the performance of your drive will make a huge difference, on the contrary.
But the difference is ony noticeable from say.. upgrading from 20GB to 200GB for example. Or 40 GB to 250GB.
Currently, top performers are 200GB +. 160GB drives are not too far behind.
Speed is also determined by platter size, so naturally larger drives tend to have larger platters.