Well, if you're using IE6, that shouldn't be an issue. So you can forget about Firefox - although I'd still try it to eliminate the possibility of something being wrong with your browser.
If the Internet light goes out when you visit your config page, this is certainly unexpected behavior. My initial assumption would be your router is failing.
Hard resetting it like Samstoned mentioned could be a good idea, but you need to be familiar with how to set up your router again so you can connect to your Internet service. Then, there's the possibility you won't still be able to get into the config page - that could put you in quite a pickle if you router doesn't connect automatically to your ISP without configuration.
To reset your Belkin router, you may have to unplug the unit, hold the reset button, plug it back in and wait about 5 seconds. That should reset it, in case your previous attempt didn't work.