Yes you will need to format you new hard drive before you can use it. You may also need to create a partition on it as well. Boot up with a Windows 98 boot floppy, type fdisk and hit enter. Yes enable large disk support, then after that you will want to create primary DOS partition on your new drive.
After that it will make you reboot, do this, still boot up from the cd. Now you can format your new drive by typing format x: where "x" is your new hard drive. Before you do this you might try to dir x: just so you be sure that this is infact your new hard drive and not your old one.
Now boot into windows, and you can use your new drive.