Yes, FAT32 is usually the solution because both Windows and Mac OS can work with it...
But FAT32 is limited by a maximum of 4GB per file. So, it may be unlikely, but if you are planning to store files larger than 4GB (Home movies etc...) then FAT32 isn't really an option for you.
If the above is a concern, then you can leave the drive formatted as HFS+ and buy Macdrive. It is a terrific little program that runs on your Windows system, allowing Windows to work with HFS drives (Mac formatted).
Your second option would be to format the drive with NTFS and use FUSE + NTFS3G for Mac OS X.
Fuse, the program that allows file system extensibility
http://code.google.com/p/macfuse/downloads/list
Ntfs-3g allows support for NTFS through Fuse
http://www.ntfs-3g.org/
Fuse GUI, a simple management program for Fuse
http://code.google.com/p/macfusion/downloads/list
And of course, FAT32 is recommended because it is well supported by both OS X and Windows. But if you NEED access to files larger than 4GB, then ntfs-3g or Macdrive are your only options. By the way, you can check the formatting of your drive in OS X by clicking on Go > Utilities > Disk Utility. In windows, you just right click on the drive icon in "My Computer" or "Computer" and click "Properties".