There are no rules set in stone, but in general, motherboards like two matched pairs of ram (both in speed and size). If you are mix-matching things, some systems become unstable, some don't. Sounds like your system doesn't like the combo you are trying.
Is one stick single-sided and another double sided? This could also be the problem. If so, take a look in your manual for the correct ram configuration (i.e, what slot you should be popping the ram in).