#1
|
|||
|
|||
excel macro returning the wrong value
Hi Folks Not sure if this has already been posted. I have set up two variables as double in an excel macro and populated them with two values from cells in the workbook. Using Msgbox I print out the variables and get the expected values of 14.1 and 14.1. I use a further Msgbox to print out the difference between the two variables (variable 1 - variable 2) and I get -1.77635683940025E-15 I was expecting this to be zero. I have tried recreating the excel file and making sure that the cells are all formatted as numbers with 1 decimal place. I have tried setting the variables to zero immediately before inserting the values. This is in a loop and other values are correct. What am I doing wrong? |
#2
|
||||
|
||||
Are the two cells calculated values or were they typed in as 14.1 exactly?
__________________
Andrew Lockton Chrysalis Design, Melbourne Australia |
#3
|
||||
|
||||
anne.kiss: Please don't add unnecessary polls to threads. I hardly think you wanted people to vote on whether someone could help...
Furthermore, your thread title shows the question concerns Excel, not Word, yet you've posted in the Word VBA forum instead of the Excel programming forum. You'll have more chance of getting help if you post in the correct forum. Poll stripped & thread moved to the correct forum.
__________________
Cheers, Paul Edstein [Fmr MS MVP - Word] |
#4
|
||||
|
||||
Perhaps you are the victim of Excel floating point arithmetic ?
__________________
Did you know you can thank someone who helped you? Click on the tiny scale in the right upper hand corner of your helper's post |
#5
|
|||
|
|||
Many thanks for these replies. The two values were calculated, not input manually.
Yes you are correct I added a poll by mistake - this was my first attempt at a post, so got it wrong. When I went into this forum today my post was under excel programming, so maybe the website changes things for you? Is there any way of getting round the floating point arithmetic? I only have one digit after the decimal point so I could split the number into two cells and do two calculations and then put the numbers back together again. This would be very messy, so if anyone knows how to make sure the floating point arithmetic is correct, please let me know. |
#6
|
||||
|
||||
Quote:
__________________
Cheers, Paul Edstein [Fmr MS MVP - Word] |
#7
|
||||
|
||||
Quote:
BTW using an email address as username is an open invitation for spammers...
__________________
Did you know you can thank someone who helped you? Click on the tiny scale in the right upper hand corner of your helper's post |
#8
|
||||
|
||||
Quote:
|
Tags |
cell value, excel macro |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Table column width wrong - unless macro paused!? | Kebap | Word VBA | 7 | 09-21-2018 03:14 PM |
Returning text, date, time, location from one Excel sheet to another | dave57 | Excel | 17 | 06-15-2017 10:45 AM |
Excel colums are wrong way around | SteveL | Excel | 1 | 05-06-2015 02:09 AM |
Macro operates on wrong document window | marceepoo | Word VBA | 1 | 12-20-2012 03:12 PM |
need macro for quiz (?) repeat of wrong answers with shuffle | braveshark | PowerPoint | 0 | 10-02-2012 06:10 AM |