![]() |
|
|
|
#1
|
|||
|
|||
|
This query is aimed at anyone interested of course, but Paul you are (or appear to be) the resident mathematician here so especially hoping you will have a solution or at least an opinion.
I have been tinkering with a process to set graphic file GPS Latitude and Longitude coordinates using VBA. I have that process worked out, but I have a question about the accuracy (or seemingly inconsistencies) of some mathematical processes. Looking at the following procedures, your will see the issue. 1. Run ShowComponents and you will see the Deg, Min, Sec and Reference values returned for a very precise GPS Latitude decimal value. 2. Next run Show issue where those same returned values are passed and a decimal value is returned. As you see, the returned value is close, but not exactly the same as the initial decimal value passed in ShowComponents. Is there anything I can modify or change to improve this accuracy? Thanks. Code:
Sub ShowComponents()
Dim varComponents
Dim lngIndex As Long
'This procedure converts a very precise GPS Latitude Decimal position to it Deg, Min, Sec eequivalent
varComponents = fcnConvertDecimalCoordToDMS2(35.1738086504322, "N")
For lngIndex = 0 To UBound(varComponents)
Debug.Print varComponents(lngIndex)
'As you see, the seconds is returned with 13 decimal places.
Next lngIndex
lbl_Exit:
Exit Sub
End Sub
Function fcnConvertDecimalCoordToDMS2(ByVal sngDecimalIn As Single, strCR As String)
Dim lngDeg As Long, lngMin As Double, sngSec As Double
Dim varVals(3)
lngDeg = Int(sngDecimalIn)
lngMin = Int((sngDecimalIn - lngDeg) * 60)
sngSec = ((sngDecimalIn - lngDeg) * 60 - lngMin) * 60
varVals(0) = lngDeg
varVals(1) = lngMin
varVals(2) = sngSec
varVals(3) = strCR
fcnConvertDecimalCoordToDMS2 = varVals
lbl_Exit:
Exit Function
End Function
Sub ShowIssue()
Debug.Print fcnConvertDMSCoordinates_To_Decimal2(35, 10, 25.7125854492188, "N")
'Should return 35.1738086504322 but returns 35.1738090566359
End Sub
Function fcnConvertDMSCoordinates_To_Decimal2(lngDeg As Long, lngMin As Long, sngSec As Single, strCR As String)
Dim lngCR As Long
Dim sngMin As Single
'Determine sign from the cardinal reference
Select Case strCR
Case "N", "E": lngCR = 1
Case "S", "W": lngCR = -1
End Select
sngMin = lngMin / 60
sngSec = sngSec / 3600
fcnConvertDMSCoordinates_To_Decimal2 = lngCR * Abs(lngDeg) + Abs(sngMin) + Abs(sngSec)
lbl_Exit:
Exit Function
End Function
|
|
#2
|
||||
|
||||
|
Try converting all your Singles to Doubles...
35.1738086504322, "N" = 35, 10, 25.7111415559194 "N"
__________________
Cheers, Paul Edstein [Fmr MS MVP - Word] |
|
#3
|
|||
|
|||
|
Paul,
Yes, that seems to work. Thank you. I realize that this isn't math lab but would you care to provide an elementary explanation of the difference when the single variables are used? |
|
#4
|
||||
|
||||
|
Hi Greg,
Ultimately, it's a question of precision. Doubles (8-byte) are more precise than singles (4-byte). In this case, singles just aren't sufficient for the accuracy required. For more, see: Single Data Type - Visual Basic Double Data Type - Visual Basic
__________________
Cheers, Paul Edstein [Fmr MS MVP - Word] |
|
#5
|
|||
|
|||
|
Paul,
Doh, I thought something, but what I thought (mistakenly of course) was that singles were more precise than doubles. Thank you. |
|
|
|
Similar Threads
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| How insert Math ML code as math in word document | dhanaraj_sri | Word VBA | 4 | 05-03-2023 10:08 AM |
| Find allowed errors when accuracy, confidence, and number of trials is known | SerenityNetworks | Excel | 2 | 10-27-2017 09:21 AM |
[Project 2013] More accuracy with resource, unit %, and cost?
|
cag8f | Project | 18 | 01-02-2016 01:51 PM |
| Roll-Up Duration: Changing Units of Measure & Also Accuracy Of | Pmacdaddy | Project | 3 | 04-20-2012 07:13 AM |
Team Planner-maintaining accuracy in task start and finish times
|
david_23 | Project | 9 | 04-19-2012 08:33 AM |