For dual chanel you need atleast matching sizes. You system can only work as fast as the worst RAM, and that's if it works at all. Compatibilty is not certain. It is best to get all the same if you can. If you have four slots full stabily can go down, too.
All this depands alot on your systems specs. I have had two matched 512mb, with two match 256mb with the same timings together once in dual chanel on my Asus P5P800. It worked fine at stock speed.