CPU are optimized for sequencial processing. It fits most of the normal application as they don't execute in parallel very well. At least they weren't though like that. CPU usually have between 4 and 16 cores for paralelization. GPU are optimized for highly parallel tasks. Usually, each task is quite small but can be executed in parallel.
The basic idea of CUDA is to help developer to make the best use of CPU and GPU. Any thing that can be processed in parallel is best processed by GPU while sequencial algorithm is best executed on CPU.