Quote:
Originally Posted by xor
When I do your calculation in Excel I get:
=41048/1312319*895.33 = 28.0050093308
|
Quote:
Originally Posted by melissapete24
I pulled out the calculator and did the math again, and it is still saying 28.004937. But, I pulled out my iPhone and used that calculator, and it came out the same as what you got, as did the calculator on the computer. [....] How the heck is a CALCULATOR wrong?
|
It is not "wrong" per se. Ironically, it is the calculator, not Excel that is "rounding twice".
More accurately, the calculator is simply rounding (or truncating) to a different number of decimal places (or significant digits).
[EDIT] And it might be a decimal calculator; that is, it might use decimal instead of binary representation internally.
We can simulate the calculator result by doing the following in Excel:
A1: 41048
A2: 1312319
A3: 895.33
A4: =ROUNDDOWN(A1/A2, 7)
A5: =A4*A3
A4 displays 0.0312789, and A5 displays 28.004937537.
Presumably, the calculator displays only 6 decimal places, truncating.
What calculator did you use? Perhaps the number of decimal places (or significant digits) is configurable.