Had the same problem, searched the Internet and found a million people with the same problem and a million helpful people and their "solutions" to the problem, of which some did just not work, and some were downright dangerous to the health of my PC. But at last I found a solution that works, and it's simple (thank you saykomatrixx):
First (of course) you check your BIOS to see that USB-devices are below (after) internal HD:s in the boot sequence, and that might be all you need to do.
If that does not work, or your BIOS (like mine) does not have this option (I only had Disk drives, Hard Drives, Network drives, and Floppy drives), then here's the beauty:
Find (in BIOS) the "USB Legacy Support", make sure it is set to "Enabled", and voilà, it should work like a charm to start-up with the external HD:s connected and on.
I did get a small surprise during next start-up when Windows started to install drivers for my sound card, and later I had to change some settings in my sound card to get my microphone to work again, as the driver install had changed some settings there, but otherwise everything is just hunky dory.
Good luck.
PS
I don't really know what the "USB Legacy Support" is/does (perhaps a more educated member could tell us?) but as my already was in "Auto" which is a sort of semi-on (software controlled is my guess), the difference can't be that huge from before I switched it fully on.
DS