It's all still there, you just can't use all of it. With win7 especially, indeed you are using some for the user interface and just running the OS in general.
You might get some back by disabling Aero and screen peek. Also remove the transparency from the taskbar. That helps a little, but having 4 gigs worth of hardware will never mean that you get to use all of it for your own programs.