VBA Math Accuracy

[gmaxey]

This query is aimed at anyone interested of course, but Paul you are (or appear to be) the resident mathematician here so especially hoping you will have a solution or at least an opinion.



I have been tinkering with a process to set graphic file GPS Latitude and Longitude coordinates using VBA. I have that process worked out, but I have a question about the accuracy (or seemingly inconsistencies) of some mathematical processes.


Looking at the following procedures, your will see the issue.


1. Run ShowComponents and you will see the Deg, Min, Sec and Reference values returned for a very precise GPS Latitude decimal value.


2. Next run Show issue where those same returned values are passed and a decimal value is returned. As you see, the returned value is close, but not exactly the same as the initial decimal value passed in ShowComponents.


Is there anything I can modify or change to improve this accuracy?


Thanks.

Sub ShowComponents()
Dim varComponents
Dim lngIndex As Long
  'This procedure converts a very precise GPS Latitude Decimal position to it Deg, Min, Sec eequivalent
  varComponents = fcnConvertDecimalCoordToDMS2(35.1738086504322, "N")
  For lngIndex = 0 To UBound(varComponents)
    Debug.Print varComponents(lngIndex)
    'As you see, the seconds is returned with 13 decimal places.
  Next lngIndex
lbl_Exit:
  Exit Sub
End Sub
Function fcnConvertDecimalCoordToDMS2(ByVal sngDecimalIn As Single, strCR As String)
Dim lngDeg As Long, lngMin As Double, sngSec As Double
Dim varVals(3)
  lngDeg = Int(sngDecimalIn)
  lngMin = Int((sngDecimalIn - lngDeg) * 60)
  sngSec = ((sngDecimalIn - lngDeg) * 60 - lngMin) * 60
  varVals(0) = lngDeg
  varVals(1) = lngMin
  varVals(2) = sngSec
  varVals(3) = strCR
  fcnConvertDecimalCoordToDMS2 = varVals
lbl_Exit:
  Exit Function
End Function
Sub ShowIssue()
  Debug.Print fcnConvertDMSCoordinates_To_Decimal2(35, 10, 25.7125854492188, "N")
  'Should return 35.1738086504322 but returns 35.1738090566359
End Sub
Function fcnConvertDMSCoordinates_To_Decimal2(lngDeg As Long, lngMin As Long, sngSec As Single, strCR As String)
Dim lngCR As Long
Dim sngMin As Single
  'Determine sign from the cardinal reference
  Select Case strCR
    Case "N", "E": lngCR = 1
    Case "S", "W": lngCR = -1
  End Select
  sngMin = lngMin / 60
  sngSec = sngSec / 3600
  fcnConvertDMSCoordinates_To_Decimal2 = lngCR * Abs(lngDeg) + Abs(sngMin) + Abs(sngSec)
lbl_Exit:
  Exit Function
End Function

[macropod]

Try converting all your Singles to Doubles...

35.1738086504322, "N" = 35, 10, 25.7111415559194 "N"

[gmaxey]

Paul,
Yes, that seems to work. Thank you. I realize that this isn't math lab but would you care to provide an elementary explanation of the difference when the single variables are used?

[macropod]

Hi Greg,

Ultimately, it's a question of precision. Doubles (8-byte) are more precise than singles (4-byte). In this case, singles just aren't sufficient for the accuracy required. For more, see:
Single Data Type - Visual Basic
Double Data Type - Visual Basic

[gmaxey]

Paul,


Doh, I thought something, but what I thought (mistakenly of course) was that singles were more precise than doubles. Thank you.