Yes, very correct! Just to make sure you've got it right though, I'll rephrase what you have said.
You will need to set the second drive to "slave" by setting its jumpers to slave mode (refer to your manual or manufacturer's website). The primary drive (I'm assuming the hard disk you already have in your system) should be set to master already, so you will not have to change the jumpers on it.
IF you have trouble,, here are some common troubleshooting tips.
1.) Double check jumper settings. If slave/master does not work, then try cable select.
2.) Ensure your cables and nice and snug
3.) Arrange the drives on the cable in a different manner.
4.) Try the new hard disk by itself on master to test if it works
5.) Try a different IDE channel
6.) Check out the BIOS and see if your settings are correct, systems made in the past few years almost always work with "auto" settings.
7.) Try a different cable
8.) Unplug other devices (such as other drives or PCI cards) to make sure the disk is getting enough power. Your PSU could be underrrated and causing strange problems.
If the disk doesn't spin when it is plugged up, you either have a bad disk or #6 is a likely cause.
If you are using Windows 98 or Me (or any Windows OS for that matter), make sure you double check whether or not the hard disk is set to "DMA mode" in device manager. Many times, it defaults to PIO mode which is woefully slow.
Enabling "IDE busmaster" in your BIOS (if available) will increase disk speed outside of the Windows enviroment usually.. I recommend leaving this on if you use things like Drive Image.
And lastly, don't hit anything with a big hammer.