I am trying to figure out if I can overclock my bad Celeron Processor. I am new to doing this and any help would be appreciated.

You're ability to overclock depends completely on your motherboard - which you failed to mention anything about. ;)

Overclocking these days is done through the BIOS setup, usually as some sort of 'advance configuration' menu. If you're using a prefab system such as HP, Sony, IBM... Then your chances of overclocking are fairly nil.

Details on overclocking
http://en.wikipedia.org/wiki/Overclocking

Here's a random overclocking article:
http://www.neoseeker.com/Articles/Hardware/Guides/OCguide/