CPUs and GPUs are different because they are tasked and built to handle different tasks and do things in a different way.
A CPU is composed of a small number of cores and has high caches designed to handle much more complex tasks (or really any computational task).
A GPU is composed of as much as 1000's of cores that are much simpler but run things in parallel to give you a high speed computational power.
In a sense, a CPU can do anything a gpu can (in some cases not as well) but not vice versa. For example, bitcoin mining is a simple hashing program that is searching through a code sequence (a hash if you will, is a complex mathematical sequence) which more cores can search these sequences therefor more cores can equal more speed.