Thread: [Solved] VBA Math Accuracy
View Single Post
 
Old 08-13-2025, 08:44 AM
gmaxey gmaxey is offline Windows 10 Office 2019
Expert
 
Join Date: May 2010
Location: Brasstown, NC
Posts: 1,621
gmaxey is just really nicegmaxey is just really nicegmaxey is just really nicegmaxey is just really nicegmaxey is just really nice
Default VBA Math Accuracy

This query is aimed at anyone interested of course, but Paul you are (or appear to be) the resident mathematician here so especially hoping you will have a solution or at least an opinion.

I have been tinkering with a process to set graphic file GPS Latitude and Longitude coordinates using VBA. I have that process worked out, but I have a question about the accuracy (or seemingly inconsistencies) of some mathematical processes.


Looking at the following procedures, your will see the issue.


1. Run ShowComponents and you will see the Deg, Min, Sec and Reference values returned for a very precise GPS Latitude decimal value.


2. Next run Show issue where those same returned values are passed and a decimal value is returned. As you see, the returned value is close, but not exactly the same as the initial decimal value passed in ShowComponents.


Is there anything I can modify or change to improve this accuracy?


Thanks.




Code:
Sub ShowComponents()
Dim varComponents
Dim lngIndex As Long
  'This procedure converts a very precise GPS Latitude Decimal position to it Deg, Min, Sec eequivalent
  varComponents = fcnConvertDecimalCoordToDMS2(35.1738086504322, "N")
  For lngIndex = 0 To UBound(varComponents)
    Debug.Print varComponents(lngIndex)
    'As you see, the seconds is returned with 13 decimal places.
  Next lngIndex
lbl_Exit:
  Exit Sub
End Sub
Function fcnConvertDecimalCoordToDMS2(ByVal sngDecimalIn As Single, strCR As String)
Dim lngDeg As Long, lngMin As Double, sngSec As Double
Dim varVals(3)
  lngDeg = Int(sngDecimalIn)
  lngMin = Int((sngDecimalIn - lngDeg) * 60)
  sngSec = ((sngDecimalIn - lngDeg) * 60 - lngMin) * 60
  varVals(0) = lngDeg
  varVals(1) = lngMin
  varVals(2) = sngSec
  varVals(3) = strCR
  fcnConvertDecimalCoordToDMS2 = varVals
lbl_Exit:
  Exit Function
End Function
Sub ShowIssue()
  Debug.Print fcnConvertDMSCoordinates_To_Decimal2(35, 10, 25.7125854492188, "N")
  'Should return 35.1738086504322 but returns 35.1738090566359
End Sub
Function fcnConvertDMSCoordinates_To_Decimal2(lngDeg As Long, lngMin As Long, sngSec As Single, strCR As String)
Dim lngCR As Long
Dim sngMin As Single
  'Determine sign from the cardinal reference
  Select Case strCR
    Case "N", "E": lngCR = 1
    Case "S", "W": lngCR = -1
  End Select
  sngMin = lngMin / 60
  sngSec = sngSec / 3600
  fcnConvertDMSCoordinates_To_Decimal2 = lngCR * Abs(lngDeg) + Abs(sngMin) + Abs(sngSec)
lbl_Exit:
  Exit Function
End Function
__________________
Greg Maxey
Please visit my web site at http://www.gregmaxey.com/
Reply With Quote