What advantages do 64-bit processors have over 32-bit ones?

There are 3 most obvious advantages of 64-bit processors over their 32-bit counterparts: extended address space, capacity increase and bigger number of general-purpose registers.
The extended 64-bit address space theoretically allows the processor to work with 16 Ebytes (2^64) of physical memory within the flat memory model. Although contemporary 64-bit processors can adress only 1 Tbyte (2^40) of memory in practice, it is still much higher than in case of 32-bit addressing. A larger amount of available memory in its turn allows you to eliminate or greatly reduce sluggish operations of data swapping to the disk.
The bigger number of registers and increase of their size allow the processor to handle large memory areas simultaneously, to handle variables and arrays more effectively and to pass function arguments in registers instead of the stack.
Keep in mind that you must port your program using the 64-bit version of the compiler and consider changes of the data model (new type capacities) to get a real performance gain on a 64-bit processor. Launching an application not adapted for the 64-bit platform might on the contrary cause great performance loss depending on the specifics of the architecture of the processor being used.
But the greatest performance gain from moving to the 64-bit platform awaits those applications which handle large data arrays - these are CAD/CAE-like programs, data managers, programs to work with digital media and applied scientific applications. The performance gain might be of several hundred percent for this kind of software.
Thus, for instance, porting the service "Oracle on Demand" by the Oracle company from the 32-bit platform to the 64-bit one allowed to reduce the server response time (350 users being working simultaneously) from 13.60 seconds to 2.35 seconds. When the FASTCOM company integrated the 64-bit platform based on Itanium 2 into their IT-infrastructure, they managed to eliminate issues caused by increased load on billing systems and get a two-fold performance gain.

References

1. Discussion at Stackoverflow.com. 32 vs 64 bit...or, what exactly does 'processing information' mean
2. Evgeniy Ryzhkov, Andrey Karpov. AMD64 (EM64T) architecture.
3. Andrey Karpov, Evgeniy Ryzhkov. Lessons on development of 64-bit C/C++ applications.
4. Andrey Karpov. Optimization of 64-bit programs.
5. Oracle White Paper. Oracle Deployment on x86-64 Linux: Best Practices for Oracle on Demand