Today I ran upon an issue with using the ROUND function in Visual Basic .net (VB.net) that I would assume that others have also found to cause them a problem.
I was attempting to get 65.3125 to round correctly up to 65.313 and every way I tried to edit the code I was getting 65.312. At first I was thinking that it was an order of operations issue, but after dissecting the code, I realized that it was not.
I was using the following code and failing every time:
I was attempting to get 65.3125 to round correctly up to 65.313 and every way I tried to edit the code I was getting 65.312. At first I was thinking that it was an order of operations issue, but after dissecting the code, I realized that it was not.
I was using the following code and failing every time:
Dim dblTotal as Double
dblTotal = ROUND(65.3125 , 3)
dblTotal = ROUND(65.3125 , 3)
Result of the above code is 65.312.
The issue with the above coding is that you cannot represent 65.3125 because it rounds a double-precision floating-point value to a specified number of fractional digits. So when using the ROUND function it uses true representation and rounds down.
Correct way to code it to make it round is using MATH.ROUND :
Dim dblTotal as Double
dblTotal = MATH.ROUND(65.3125 , 3, MidpointRounding.AwayFromZero)
dblTotal = MATH.ROUND(65.3125 , 3, MidpointRounding.AwayFromZero)
Result of the above code is 65.313.
The reason the above code works is it rounds a double-precision floating-point value to a specified number of fractional digits. A parameter specifies how to round the value if it is midway between two numbers.
I hope this helps someone out a bit and saves them a bit of time. I know it helped me out greatly.
If you like this post and want to see more, follow me on my website www.chadcompton.com
Or if you prefer...
No comments:
Post a Comment
Drop me a line.