Ok I am just guessing here since its near bed time, but I would imagine that its a max of 512MB for each DIMM slot on your board, meaning a max of 1.5 GB (assuming 3 slots).... Are you sure its combined maximum and not max per slot???? (again I have not checked because its beddy-bye time for me soon...)
Operating system have certain max limits as well, but with XP professional I believe its 4 GB so I think you are safe on the OS front, its just the hardware we must now consider.
You must be doing something fairly trippy though if you need over 512 MB of RAM, unless you are doing some serious number crunching, graphics work, database work, etc.
The domain controller at my work has like 512MB and its contending with entertaining like 2000 odd users so I think that over that amount is, at this moment in time for the industry, a little excessive.... (Again unless you are doing certain graphical work, etc.... )
Otherwise, I am inclined to think that any slowness you are experiencing is the result of misconfigurations / lack of tweaking on both hardware and software levels in your PC. I have 512 MB of RAM in my server, and although I would like to upgrade soon to 1 GB, more or less just to show off to people, its not something high on my agenda right now.
512 MB should be fine for XP so maybe there's something else going on here....
Its doubtful you will get any joy with more RAM installed in your machine than your mobo can handle.... It may POST and then again it might not.... Maybe post with just the max RAM it can address.... There might be a BIOS update to fix that but that's doubtful.
My main observations would be:
1)Why the hell do you want more than 512 MB anyway? (I will be perfectly receptive to a good answer....)
and
2)No, more than your motherboard manual states will almost certainly not work.