IEEE 754 Standard for Floating Point Binary Arithmetic

There are two main ways to represent real numbers in binary. Fixed point and floating point. I've talked about these in detail in other videos, but to quickly remind you, fixed point representation fixes the position of the binary point within a register, and therefore fixes the sizes of the whole number and the fractional parts of any real number stored in that register. Place values to the left of the point are bigger than 1, and place values to the right are smaller than 1. Each place value is double that of the place value on its right. Exactly where the point is fixed depends on the way that that particular computer system has been designed. Fixed point binary can greatly simplify the processing of real numbers, but it imposes big limitations on the range and precision of the real numbers that can be stored. Floating point representation is the most common way of encoding real numbers in binary. It uses a binary version of the standard scientific notation, with a base number, the so-called mantissa, and an exponent. The floating point format covered in my other videos assumes there's an imaginary binary point located immediately to the right of the leftmost bit in the mantissa, the so-called sign bit. The value of the exponent specifies how far to the left or to the right the binary point should float in order to obtain the actual value in the register. The number of bits available for the exponent governs the range of the number that can be represented, that is, how big or how small, and the number of bits available for the mantissa governs the precision, that is, the accuracy of the number. This floating point format employs 2's complement to represent negative values in both the mantissa and the exponent. Historically, floating point formats have varied from computer to computer. For example, the number of bits assigned to the mantissa and the exponent could vary. For a fixed-sized register, more mantissa bits means better precision, but this comes at the expense of exponent bits, so better precision means a smaller range. Some formats would place the mantissa bits to the right of the exponent bits, and some would place them on the left, as shown here. Some formats would use sign and magnitude to represent negative values, while others would use 2's complement. It wasn't until 1985 that the Institute of Electrical and Electronics Engineers defined a standard format for floating point binary, a format that is used in most of today's computers. So, in this video, I'll describe the IEEE 754 standard for representing floating point binary numbers. The IEEE 754 standard defines the layout of a 32-bit single-precision floating point number, as shown here. If you're a programmer, you may be familiar with the variable data types single and double. Online help about these data types includes the range of values that can be stored. I'm going to focus on single precision for now. Notice that the sign bit is the leftmost bit, followed by an 8-bit exponent, and then a 23-bit mantissa. The mantissa actually has 24-bit precision, but only 23 bits need to be stored. More about that in a moment. Double precision also has a single sign bit on the left, followed by an 11-bit exponent, and then a 52-bit mantissa, giving a total of 64 bits instead of 32. Let's look at an example. I'm going to convert the denary value, 19.59375, into IEEE 754 standard single-precision 32-bit floating point binary. The first step is trivial. Determine the sign bit. If the number is positive, the sign bit will be 0. If it's negative, the sign bit will be 1. This is a positive number, so the sign bit is 0. In the second step, convert the denary value into pure binary, in other words, convert it into a fixed-point format. At this stage, you can treat negative numbers as if they were positive, because you've already established the sign bit. For an uncomplicated number like 19.5 or 19.25, you can easily determine the place values you need to make up the number. But 19.59375 is not obvious. A common approach involves repeatedly halving the whole number part and repeatedly doubling the fractional part. 19 divided by 2 is 9 with a remainder of 1. 9 divided by 2 is 4 with a remainder of 1. 4 divided by 2 is 2, remainder 0. And 2 divided by 2 is 1 with a remainder of 0. And there's 1 remaining at the end. If you think about it, there's always going to be 1 at the end. The remainders give you the bit pattern for denary 19. To convert the fractional part, you repeatedly multiply it by 2. 0.59375 times 2 is 1.1875. The 1 in front of the decimal point of this result is the first binary digit you need. The fractional part of this result is then multiplied by 2. 0.1875 times 2 is 0.375. The result starts with 0, which is your next bit. 0.375 times 2 is 0.75. Another 0 is generated. 0.75 times 2 is 1.5, so this time you have a 1. And finally, 0.5 times 2 is 1. This is when to stop, when the fractional part is 0. This method of converting from denary to binary is particularly good for spotting a recurring sequence of bits, as you'll see later. Now you need to normalise the pure binary representation. The binary point is moved so that only one non-zero bit remains to the left of it. In this case, the point is repositioned four places to the left. To get the point back to where it was, you would have to float it four places to the right. So now you have the value of the exponent, namely 4. The normalised binary value is therefore In step 4, you need to calculate the biased exponent. If you're simply following a recipe, all you'll need to do is add 127 to the value. In this case, you'll get 131, and then convert that into an 8-bit unsigned binary integer. I'll explain the principle and reasoning behind the biased exponent in just a moment, but for now, let's finish off this conversion. The final step is to remove the leading 1 from the mantissa. The leftmost bit of the mantissa is always going to be 1 for any real number that we want to represent in this format, so there's no need to store it. We know it belongs there, and that it needs to be put back before doing any calculations with this number. In actuality then, this format has a 24-bit mantissa, but we only need to store 23 bits of it. All that remains to be done now is to pad out the mantissa with trailing zeros. Now, let's say a little bit more about the way the exponent is encoded. Why do you need to add 127, and what's the benefit of this approach? Suppose for a moment that you have a 4-bit exponent. It's easier to see how this works with only 4 bits. Exactly the same principle applies to an 8-bit exponent, or indeed, an exponent with any number of bits. With 4 bits, you can see that we have 2 to the power of 4, that's 16 different combinations of 1's and 0's. So a 4-bit exponent can have one of 16 possible values. If you only ever needed to represent positive exponents, then you could represent all of the values from 0 to 15. These bit patterns are easily converted into unsigned integers with a simple place value gadget. But, since you also need to represent negative values, you have to sacrifice some of the positive values. One way to do this is to use 2's complement, and you can use a modified gadget to do the conversions. The most significant bit has a negative place value. 2's complement allows for an equal number of positive and negative values, assuming that 0 is a positive value. Notice how all of the negative values begin with 1, and all of the positive values begin with 0. This is why the leftmost bit is called the sign bit in 2's complement. Now, suppose you wanted more positive values than negative values. This requires a different approach. Here, you can see there are only 5 negative values, but 10 positive values. Notice that the denary value, minus 5, is represented with four 0's, and that the denary value, 0, is represented with the bit combination 0101. Because of this peculiarity, you can no longer convert directly between binary and denary. You need to perform an additional step. Suppose you wanted to find the bit pattern that represents, say, denary 3. You need to add 5 to 3 first, because there are 5 negative values. And then you can convert the result into an unsigned binary integer. 5 plus 3 is 8, and 8 in unsigned binary is 1000. To find the bit pattern for, say, minus 3, you should employ exactly the same method. Add 5 to minus 3, which is 2, and 2 in unsigned binary is 0010. If you want more negative values than positive values, you can allocate the bit patterns differently. In this case, there are 9 negative values and 6 positive. To find the bit pattern for, say, denary 4, you need to add 9 first. 9 plus 4 is 13, and the unsigned bit pattern for 13 is 1101. The same system is being used here, but this time there are an equal number of positive and negative values. To get the bit pattern for any one of these values, you first need to add 8 to the number that you want to encode, because that's how many negative values there are. This final set favours positive numbers. This is actually how the IEEE 754 standard works. With 4 bits available, there are 2 to the power of 4 minus 1, that's 8. Minus 1, that's 7 negative values. So 7 is the number that you need to add to any value that you want to encode before converting it into unsigned binary. The value that you need to add is called the offset, or the bias. With an 8-bit exponent, which is the case for a 32-bit single precision real number in IEEE 754 format, the bias is 2 to the power of 8 minus 1, minus 1, which is 127. For a double precision real number, which has an 11-bit exponent, the bias is 2 to the power of 11 minus 1, minus 1, which is 1023. There's even a 128-bit quant precision format. The same formula applies. The big advantage of a biased exponent is the way that the values of different exponents can be compared easily. This is by virtue of the fact that the binary encodings are in lexical order, as are the denary values that they represent. Here are my 15 4-bit combinations, and I'm going to jumble them up. Now, watch what happens when I sort them as if they were text. This means the computer can easily tell if one number is bigger than another by comparing them bit by bit. This is not possible if two's complement is used. Here are two examples that you might like to try yourself. Pause the video to give them a go, and I'll show you the solutions in a moment. The second example highlights a couple of important issues. And here are the solutions. For the first example, determine the sign bit. This is a positive number, so the sign bit is zero. In step two, we convert the value into pure binary. This is what it looks like. Let's see how we got there. To get the whole number part, well, easy. Zero divided by two is zero, remainder zero. To get the fractional part, we've repeatedly multiplied by two. Remember to read the fractional bits from top to bottom. Step three is to normalise so we can get the mantissa and the unbiased exponent. Our binary value becomes 1.1 times 2 to the minus 4. That's how many places to the left we'd have to move the binary point to get the original value. Now we determine the biased exponent. In this case, minus 4 plus 127. That's 123 in base 10, and when converted into 8-bit binary, we have the bit pattern you can see here. In step five, remove the leading 1 from the mantissa, so we simply have 1. And finally, you can assemble the result. We've padded out the mantissa with zeros to make it up to 23 bits. In the second example, the sign bit is a 1 because it's a negative number. When we convert this into pure binary, this is what we get. Now I'll show you how we got there, but first of all, let me point out that the last four bits are recurring. That's why they've been overscored and there are three dots at the end. To get the whole number part, we divide by 2 repeatedly. That was pretty straightforward. Remember to read the bit pattern upwards for the whole number part. To get the fractional part, you repeatedly multiply by 2. Now, as you're doing this, you'll notice that the value 0.6 comes up again, in which case you have a recurring bit pattern. There's no need to keep going. And remember to read this sequence of bits from top to bottom. Now we normalise to get the mantissa and the unbiased exponent. The point has to move 6 places to the left in order to follow the leftmost 1. An exponent of 6 now needs to be converted into a biased exponent, so we add 127, which is 133, and that is converted into an 8-bit unsigned binary integer. The leading 1 is then removed from the mantissa, and we make the mantissa up to 23 bits. Now, there's an extra step that you could perform at this point. The mantissa includes a recurring bit sequence, which means it's too long to be stored with a finite number of bits. You might choose not to do anything in particular about it and simply chop the mantissa at 23 bits to make it fit. If you check your result with an online calculator, you might see that the mantissa has simply been chopped. Simply chopping the mantissa is effectively rounding it down. But in reality, a computer will round the mantissa either up or down to the nearest value. Some online calculators will round the result properly. If you want to round your mantissa properly, ask yourself what the 24th bit would have been if there was room to store it. If it would have been a 0, you can simply chop. If it would have been a 1, which is the case here, then you should add 1 to the mantissa. This is a regular binary addition operation, so it might generate carry bits that need to be dealt with. When it comes to rounding, there is a special case In this case, if the 23rd bit that you are able to store is a 1, then you add 1 to the mantissa. If the 23rd bit is 0, then you chop. I'll talk more about rounding and rounding errors in another video. The final job is to assemble the result. Here are two conversions in the other direction that you can try if you wish. Solutions to these in a moment. And here are the solutions. For the first example, determine the sign in denary. The sign is positive because it starts with a 0. Then, determine the exponent in denary. The exponent bits represent the value 132. Now we need to remove the exponent bias. So that means subtracting 127. We have an exponent of 5. Convert the mantissa to denary. The significant bits of the mantissa have been put into this place value gadget. This gives us the value 0.828125. Now we add 1 to the mantissa and we include the sign. And then we can calculate the final result. We have 1.828125 times 2 to the power of 5. If you do this on the calculator, you'll get 58.5. Calculate 2 to the power of 5 first and then multiply this by the other value. In the second example, the sign of the number is negative because the sign bit is a 1. Now, determine the exponent in base 10. In this case, it's 9. We need to remove the exponent bias, which means subtracting 127. So we have an exponent of minus 118. Then convert the mantissa into denary. Plugging the binary digits into the place values gives us 0.09765625. Add 1 to the mantissa and include the sign. And we can calculate the final result. We want the result in base 10, so we have to multiply minus 1.09765625 by 2 to the power of minus 118. This will give you a purely base 10 result, which you can see is rather small. You can leave the result like this if you wish, or you can float the point to the left. But you'll have to think about doing some base 10 rounding at the other end. To finish off, I'll briefly mention a couple of exponent values that have special meanings, depending on the value of the mantissa. If the exponent bits are all 1s and the mantissa is all 0s, the number being represented is infinity. If the exponent bits are all 1s but the mantissa is not all 0s, then the value being represented is not a number. If the exponent bits are all 0s and the mantissa is all 0s, then the value being represented is indeed 0. If the exponent bits are all 0s and the mantissa is not all 0s, the value being represented is very, very small. This is known as a subnormal or denormalised number. These are the topic of another video.