Why do 64-bit applications work faster than 32-bit ones?

The difference in performance between 32-bit and 64-bit versions of applications greatly depends upon their types and data types they are processing. But in general you may expect a 2-20% performance gain from mere recompilation of a program – this is explained by architectural changes in 64-bit processors [1].
More general-purpose registers in such processors let you optimize work with local variables in functions which do not need to be saved into main memory anymore. Several function arguments are passed through registers and it also reduces the time to call them [2].
The extended address space eliminates the limitation concerning the maximum 4 Gbytes of main memory available on the 32-bit architecture. The possibility to store the whole necessary data set in the main memory allows to avoid too slow operations of data swapping to the disk and it lets some programs working with large data arrays get a performance gain in several times.
However, the other side of the move to the 64-bit version is a two-time increase of the sizes of pointers and some other data types that might result in an increased demand of the software to the system’s physical memory. In some cases, it might slow down the speed of a 64-bit application in comparison to a 32-bit one. However, it occurs rarely and such cases are usually determined by an unsuccessful choice of the format in which data are stored in the program.
Note also that when you launch 32-bit versions of software on 64-bit systems of the Windows family, old 32-bit applications are executed a bit slower because of the WoW64 subsystem that emulates the 32-bit environment. An average performance loss because of this WoW64 layer is 2-3% although in some special cases it might be much more.

References

1. Andrey Karpov. Optimization of 64-bit programs.
2. Andrey Karpov. OOO "Program Verification Systems" company blog. The reasons why 64-bit programs require more stack memory.
3. Discussion at Stackoverflow.com. Are 64 bit programs bigger and faster than 32 bit versions?