Microsoft Office Forums

Go Back   Microsoft Office Forums > >

Reply
 
Thread Tools Display Modes
  #1  
Old 06-04-2014, 08:42 AM
required_username required_username is offline Rounding values inconsistent Windows 8 Rounding values inconsistent Office 2010 32bit
Novice
Rounding values inconsistent
 
Join Date: May 2013
Posts: 28
required_username is on a distinguished road
Default Rounding values inconsistent


Hope this is easy.
I have a series of calculations and appear to be having trouble with rounding. I'll do my best to explain.

Cells C2 through C7 are multiplied by a different 'static' number and rounded to 2 decimal places. I then add C2 through C7 and put the sum in C9.
With frequency, the total showing in C9 is NOT equal to the values shown in C2 >C7. If the numbers don't match, the difference is always 0.01. I suspect this is due to the rounding. How do I get the value in C9 to give an accurate value? Or, is the C9 value correct but one of the values in C2>C7 wrong?
Reply With Quote
  #2  
Old 06-04-2014, 10:49 AM
gebobs gebobs is offline Rounding values inconsistent Windows 7 64bit Rounding values inconsistent Office 2010 64bit
Expert
 
Join Date: Mar 2014
Location: Atlanta
Posts: 837
gebobs has a spectacular aura aboutgebobs has a spectacular aura about
Default

I think the problem is that the cells in C are not really rounded, only their display is limited to some number of decimals. If you have a cell that has a value of 3.14159265358979, when displayed with just two decimals, it appears to be 3.14 though the actual value stored is 0.00159265358979 greater.

This is a disconnect between the spreadsheet world and the world of science where it is necessary to mind significant figures of precision in order to calculate results that most accurately reflect that precision. It also presents problems such as what you are facing though few people worry about a 0.01 difference. It can be confusing if you are presenting numbers that should add to a known figure, say 100.00, and the total comes up as 99.99.

You might want to add another column that rounds the numbers in C to a specified number of digits and then add those. I'm pretty sure that will do the trick.

=round(c2,2)
Reply With Quote
  #3  
Old 06-04-2014, 11:06 AM
required_username required_username is offline Rounding values inconsistent Windows 8 Rounding values inconsistent Office 2010 32bit
Novice
Rounding values inconsistent
 
Join Date: May 2013
Posts: 28
required_username is on a distinguished road
Default

Thanks for the reply. Makes sense.
I'm a little confused on the solution.
As I understand this, I put
=round(c2,2) in, say, D2
=round(c2,3) in, say, D3, etc
is that correct? I suppose D2 'sees' the value in C2 and changes it automatically?
Reply With Quote
  #4  
Old 06-04-2014, 11:24 AM
BobBridges's Avatar
BobBridges BobBridges is offline Rounding values inconsistent Windows 7 64bit Rounding values inconsistent Office 2010 32bit
Expert
 
Join Date: May 2013
Location: USA
Posts: 700
BobBridges has a spectacular aura aboutBobBridges has a spectacular aura about
Default

I agree with your diagnosis, required—it has to be a rounding issue—and with gebobs' suggestion. But I'll expand on it a bit in case there's still confusion: As a matter of picky terminology, I would say that the value in C9 is accurate, just not precise.

Take an example:
Code:
3.14159265359
2.71828182846
2.99792458000
1.41421356237
Their precise sum is 10.27201262442, but unless you tell Excel to use special formatting all it'll show you is 10.27201262. That answer is accurate, it's just not precise to all 13 significant digits.

Now tell Excel to display them to just one decimal point, like this:
Code:
3.1
2.7
3.0
1.4
If these were the numbers' true values, the total should be 10.2, but in fact Excel claims their sum is 10.3—and Excel is correct. When you display them using just one significant digit, the display says 3.1 but the value behind the scenes, the one that's being summed, is still 3.14159-etc.

What do you do about it? Well, it depends on your needs. It would probably be more accurate for many purposes to leave it as it is, giving 10.3 as the sum rather than 10.2. But if you need to for some reason, you can round the individual numbers to their truncated form, and the sum of those values (rather than the real ones) will come out to 10.2 as you expected.

Extra credit for identifying the sample numbers above.
Reply With Quote
  #5  
Old 06-04-2014, 11:36 AM
gebobs gebobs is offline Rounding values inconsistent Windows 7 64bit Rounding values inconsistent Office 2010 64bit
Expert
 
Join Date: Mar 2014
Location: Atlanta
Posts: 837
gebobs has a spectacular aura aboutgebobs has a spectacular aura about
Default

Quote:
Originally Posted by BobBridges View Post
3.14159265359
2.71828182846
2.99792458000
1.41421356237

Extra credit for identifying the sample numbers above.
Pi
e
c (without the *10^8)
sqrt(2)

What do I win?!!
Reply With Quote
  #6  
Old 06-04-2014, 11:37 AM
gebobs gebobs is offline Rounding values inconsistent Windows 7 64bit Rounding values inconsistent Office 2010 64bit
Expert
 
Join Date: Mar 2014
Location: Atlanta
Posts: 837
gebobs has a spectacular aura aboutgebobs has a spectacular aura about
Default

Quote:
Originally Posted by required_username View Post
Thanks for the reply. Makes sense.
I'm a little confused on the solution.
As I understand this, I put
=round(c2,2) in, say, D2
=round(c2,3) in, say, D3, etc
is that correct? I suppose D2 'sees' the value in C2 and changes it automatically?
In a nutshell, yes!
Reply With Quote
  #7  
Old 06-04-2014, 12:14 PM
BobBridges's Avatar
BobBridges BobBridges is offline Rounding values inconsistent Windows 7 64bit Rounding values inconsistent Office 2010 32bit
Expert
 
Join Date: May 2013
Location: USA
Posts: 700
BobBridges has a spectacular aura aboutBobBridges has a spectacular aura about
Default

Quote:
Originally Posted by gebobs View Post
Pi; e; c (without the *10^8); sqrt(2)

What do I win?!!
My undying respect .
Reply With Quote
  #8  
Old 06-04-2014, 12:51 PM
required_username required_username is offline Rounding values inconsistent Windows 8 Rounding values inconsistent Office 2010 32bit
Novice
Rounding values inconsistent
 
Join Date: May 2013
Posts: 28
required_username is on a distinguished road
Default

Thanks for the explanation.
In my case, the values are $0.00 so exact is a necessity. I'll try "ROUND" and see if it solves my problem. I would expect I can avoid teh middle man and have the value in C2 calculated as =ROUND(R7*J2+J5,2). Correct?

As for the values, #1 is Pi, #4 is 2squared, number 2 . . . I'm not sure. I think it's 'e' but that's an educated guess. @3? No idea. Doesn't look familiar.
Reply With Quote
  #9  
Old 06-04-2014, 01:08 PM
gebobs gebobs is offline Rounding values inconsistent Windows 7 64bit Rounding values inconsistent Office 2010 64bit
Expert
 
Join Date: Mar 2014
Location: Atlanta
Posts: 837
gebobs has a spectacular aura aboutgebobs has a spectacular aura about
Default

Quote:
Originally Posted by BobBridges View Post
My undying respect .
Nice! That and $4.29 will get me an overpriced Starbuck's coffee.
Reply With Quote
  #10  
Old 06-04-2014, 07:13 PM
BobBridges's Avatar
BobBridges BobBridges is offline Rounding values inconsistent Windows 7 64bit Rounding values inconsistent Office 2010 32bit
Expert
 
Join Date: May 2013
Location: USA
Posts: 700
BobBridges has a spectacular aura aboutBobBridges has a spectacular aura about
Default

Quote:
Originally Posted by required_username View Post
In my case, the values are $0.00 so exact is a necessity. I'll try "ROUND" and see if it solves my problem. I would expect I can avoid teh middle man and have the value in C2 calculated as =ROUND(R7*J2+J5,2). Correct?
Sure, you can. All it cares is that it has some kind of numeric value to work with. Sometimes in my anxiety to boil things down to as few steps as possible I combine so many things into one that it gets much harder to understand; in such a case if I overdo it, it's sometimes better to break it up into pieces. But you're not near that threshold here..
Reply With Quote
  #11  
Old 06-05-2014, 05:35 AM
required_username required_username is offline Rounding values inconsistent Windows 8 Rounding values inconsistent Office 2010 32bit
Novice
Rounding values inconsistent
 
Join Date: May 2013
Posts: 28
required_username is on a distinguished road
Default

Very good. Tweaked it a little, but worked.
Reply With Quote
Reply



Similar Threads
Thread Thread Starter Forum Replies Last Post
Rounding values inconsistent Inconsistent Formatting YEG_V Word Tables 3 05-08-2013 03:06 PM
Custom formatting code - rounding problem venkys4u Excel 1 08-14-2012 07:45 PM
Rounding values inconsistent Standard Dev. Rounding - Excel judicial85 Excel 1 08-28-2011 05:41 PM
Inconsistent Margins judyfogarty Word 0 08-01-2010 02:54 AM
Rounding values inconsistent Federal financial rounding? markg2 Excel 6 05-26-2010 09:54 AM

Other Forums: Access Forums

All times are GMT -7. The time now is 12:06 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.
Search Engine Optimisation provided by DragonByte SEO (Lite) - vBulletin Mods & Addons Copyright © 2024 DragonByte Technologies Ltd.
MSOfficeForums.com is not affiliated with Microsoft