Why use OpenEXR?
OpenEXR allows storage of arbitary channels in a single file.
For example, you can store z depth, specular, diffuse and other channels in a single file, making asset management that much more streamlined.
32 bit vs 16 bit vs 8 bit.
This refers to the number of bits to store the information on each channel. A normal RGB image uses 8 bit. A 32 bit image uses 4 times as much memory to store compared to an 8 bit image.
Why use higher bit format (32bit instead of 8 bit)?
Higher bit format allows a much finer gradation in the value of each channels. Although our eyes and most display devices may not be able to distinguish these gradations, they become very important when used in calculation during compositing and other image manipulation processes.
An low bit format can be prone to posterization and loss of details during these processes.
What is the difference between Floating Point (FP) and Integer format?
It refers to the way the value is encoded in each channel. For example, an 8 bit integer stores value from 0 to 2^8 (256) in discreet numbers.
A floating point format stores values from 0 to 1 but in rather different system from integer. For example, an 8 bit floating point may allow values with a difference of more than 256. This is very important in HDRI situation when one pixels in an image can be 1000000 times brighter than another pixel.
The practical implication is that integer allows a more precise calculation while floating point allows greater range. It also means that integer is the preferred format for final storage while floating point is the preferred format for preprocess storage.
What is half-precision floating point?
It is a particular type of floating point format that allows 16bit channels to represent the information that is normally contained in 32bit floating point channels. It means it need only half the storage space of 32bit floating point format.
However, keep in mind that the consolidation in EXR format is still ongoing. Not all the major image editing/compositing softwares handle all the various formats well or in the same way.
More details at wikipedia.
Feel free to comment and correct me below. I will update this post as my understanding of the topic improves.