Slow "comparison/replace" script

CoolBlue · #1 06-20-2014, 06:36 AM

Maybe this will do it for you...

Create a Custom Class

Code:

'Class2
Option Explicit

Public i As Long
Public j As Long
Public k As Long

Then add to the standard Module with other XXXIsGone code

Code:

Const clsBytes As Long = 32 * 4
'///////////////////////////////////




'///////////////////////////////////
Sub ExampleForClass()
Dim objCls As Class2
Dim str As String
     
    Set objCls = New Class2
    objCls.i = &HAAAAAA
    objCls.j = &HABABAB
    objCls.k = &HBCBCBC
    
    lng_ObjPtr = ObjPtr(objCls)
                     
    str = PionterToSomething("objCls", lng_ObjPtr, clsBytes)
    Debug.Print str

    'Set objCls = Nothing
End Sub
'///////////////////////////////////


'///////////////////////////////////
Sub classIsGone()
    Debug.Print
    ExampleForClass
    DoEvents
    printPionterToSomething "objCls", lng_ObjPtr, clsBytes
End Sub

And try with and without the set to nothing...
No difference.

whatsup · #2 06-20-2014, 10:55 AM

Hi CoolBlue

Quote:

In order for it to work as, you need to figure out the structure of the range object in your example and reserve a buffer with sufficient bytes, for Mem_Copy to dump into.

Yes that's exactly the problem. I did try the same way as you showed, and in case of your last example it can be done, because it's simple and you can calculate the required bytes. But usually it's unpredictable the requirement of an object to copy, and therefore it won't work, - that way.

Now I gave some more thoughts to the function ObjectFromPointer(). That's because, what do I gain by copying the contents assigned to a pointer? Sure if still pointing to the same object (as in the example with objRange) I can ask if "Nothing" or use a property of the initial object to identify it. But in case the pointer is already assigned something else it will give me a hard time to keep the macro out of trouble.
Therefore I wonder if it would make sense to read always a fix parameter of length for example 32 and just compare the results. If they show a difference, it will tell that something else got assigned to the pointer, and that's good enough.

Just to avoid missunderstanding: I didn't say, that the reference count doesn't work at all. It's just as Bob put it "there might be circumstances when it doesn't".
In the moment we do with simple structures because it's about figuring out a way to keep track on ongoing things within memory. Therefore I'm expecting reference count doing allright and would be dissapointed if failing (though creating a new instance of an object was said in former times might be already to much for automatic deallocation, the same as for not handled errors).

CoolBlue · #3 06-20-2014, 04:17 PM

Quote:

Originally Posted by whatsup

Hi CoolBlue

Yes that's exactly the problem. I did try the same way as you showed, and in case of your last example it can be done, because it's simple and you can calculate the required bytes. But usually it's unpredictable the requirement of an object to copy, and therefore it won't work, - that way.

Yep, I assume that the Any Type of the parameters of your MEM_Copy declaration will typecast any Object as a pointer to that object, so

Code:

Mem_Copy oTemp, lPtr, 4

Is like

Code:

Mem_Copy ObjPtr(oTemp), lPtr, 4

The problem is that you are overwriting the first 4 bytes (in this case) of an Object type variable structure with the first 4 bytes of a Range type variable structure, coz IPtr is a pointer to the base address of a Range Object.
For sure, this will corrupt the structure of oTemp.
Then when you do

Code:

Set ObjectFromPointer = oTemp

Excel will crash coz it doesn't find what it needs at the base address of oTemp because its corrupted already.

But I'm interested that you say you said you can calculate the required bytes... How do you do that? I've been searching for an explanation of the byte structure that VBA uses for storing it's various types: do you have that?

Quote:

Originally Posted by whatsup

Now I gave some more thoughts to the function ObjectFromPointer(). That's because, what do I gain by copying the contents assigned to a pointer? Sure if still pointing to the same object (as in the example with objRange) I can ask if "Nothing" or use a property of the initial object to identify it. But in case the pointer is already assigned something else it will give me a hard time to keep the macro out of trouble.
Therefore I wonder if it would make sense to read always a fix parameter of length for example 32 and just compare the results. If they show a difference, it will tell that something else got assigned to the pointer, and that's good enough. .

Yep, I had the same thought. Did you try my example with a Class Object? I arbitrarily set the bytes count to 32*4 just to get a look at the structure, to search for the value I set the properties to (hex AAAAAA, or something like that) and you can see it clearly, a little way into the buffer. Try it and you will see what I mean.
But anyway, it seems like very start of that structure is always set to zeros when the variable is erased by VBA's memory management. So you can always see that the object has been erased. This is the best indication I've seen so far to show the life cycle of the variable.
Here are a few sample runs...

Code:

objCls : 0x18E11A58 : 0x04C0031500000000741AE1181C753904047539040000000000000000ECEC831A0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC00F4E9101500000000000000009C0601005FB4AF2E060100802300E7719425E771
objCls : 0x18E11A58 : 0x0000000000000000741AE11800000000000000000000000000000000ECEC831A0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC00F4E9101500000000000000009C0601005FB4AF2E060100800D00530041004100

objCls : 0x18E11A58 : 0x04C0031500000000741AE1181C753904047539040000000000000000ECEC831A0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC00F4E9101500000000000000009C0601005FB4AF2E060100802300E7719425E771
objCls : 0x18E11A58 : 0x0000000000000000741AE11800000000000000000000000000000000ECEC831A0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC00F4E9101500000000000000009C0601005FB4AF2E060100802300E7719425E771

objCls : 0x18E11A58 : 0x04C0031500000000741AE1181C753904047539040000000000000000ECEC831A0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC00F4E9101500000000000000009C0601005FB4AF2E060100804400E7719425E771
objCls : 0x18E11A58 : 0x0000000000000000741AE11800000000000000000000000000000000ECEC831A0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC00F4E9101500000000000000009C0601005FB4AF2E060100801800E7719425E771

And he you can see the reference counting doing its job...

Code:

objCls : 0x18E11950 : 0xEC2C0915000000006C19E118648948044C8948040000000000000000ECEC831A0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC000000000004E9101500000000000000003EB4AF2EE50000880000000000000000
after exit...
objCls : 0x18E11950 : 0x00000000000000006C19E11800000000000000000000000000000000ECEC831A0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC000000000004E9101500000000000000003EB4AF2EE50000880000000000000000

objCls : 0x18E11950 : 0xEC2C0915000000006C19E118648948044C8948040000000000000000ECEC831A0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC000000000004E9101500000000000000003EB4AF2EE50000880000000000000000
Set to Nothing...
objCls : 0x18E11950 : 0x00000000000000006C19E11800000000000000000000000000000000ECEC831A0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC000000000004E9101500000000000000003EB4AF2EE50000880000000000000000
after exit...
objCls : 0x18E11950 : 0x00000000000000006C19E11800000000000000000000000000000000ECEC831A0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC000000000004E9101500000000000000003EB4AF2EE50000880000000000000000

As you can see, the first 8 bytes is always cleared to zeros.

Quote:

Originally Posted by whatsup

Just to avoid misunderstanding: I didn't say, that the reference count doesn't work at all. It's just as Bob put it "there might be circumstances when it doesn't".
In the moment we do with simple structures because it's about figuring out a way to keep track on ongoing things within memory. Therefore I'm expecting reference count doing allright and would be dissapointed if failing (though creating a new instance of an object was said in former times might be already to much for automatic deallocation, the same as for not handled errors).

AH, OK, fair enough. Yep, sure. Well I think we have a detection tool that works now, to a reasonable level of confidence, so the next thing I was thinking to do is to create some structures that will not be released - like a Class with a circular reference for example - and see how that behaves. I'll also try crashing out of the sub on an error to see if that doesn't release the variable as well.

whatsup · #4 06-20-2014, 07:13 PM

I should have said: I can imagine you can calculate the required bytes

That is if you've got a simple object and some insight on how objects are stored in memory.

But forget about this remark, I hope we don't have to go down this road all the way.

The crash with Mem_Copy oTemp, lPtr, 4 when checking afterwards is still some secret to me. (though I might have an idea where it comes from, but I might be wrong so I will put this aside for the moment, furthermore it's not important in the moment).

Of course I tried your example with the Class (ever since I trip over your "pionterToSomething" - I should have corrected it but I didn't). What I did change, was "name as string" to "strname as string" - you know why!
And, yes, I'm familiar as to the results showing.

But let's few our new "tool" - we still facing at least two problems:
1. An ordinary object requires only 4 bytes in memory. If I'm not mistaken, every set object to an object that already exists (without using the keyword "New") only is a number in the range of "As Long". But see the example it makes it clear:

Code:

Private Const coBytes As Long = 4
  
 Sub PointsToAnotherObject()
Dim objwks As Worksheet
    Debug.Print
    Debug.Print ObjPtr(Sheet1)
    pionterToSomething "Sheet1  ", ObjPtr(Sheet1), coBytes
Set objwks = Sheet1
    Debug.Print ObjPtr(objwks)
    pionterToSomething "objwks  ", ObjPtr(objwks), coBytes
    
End Sub

So in case you store ObjPtr(objwks) in a variable and check afterwards you might think the referencecount didn't work, because it's still there (unless you remove Sheet1 from your workbook). So there won't be any hint whether or not the 4 Bytes are freed, because afterwards you always will find the object of origin.
2. If memory is freed, we might find "cleared to zero", but there might be something else assigned to the pointer. Up to now it only occured when testing with pointers to a Long-variable, but I think it always can happen. Therefore we better compare - keeping in mind point 1

Just got the message of your new post, I will have a look at it now. But I anyway wanted to say, circular reference in classes - I don't know if you can free memory at all - without closing excel.

whatsup · #5 06-20-2014, 08:20 PM

The SingleObject example:
In the error-cases the pointer is assigned to something else. By the look of it, I tend to say the error-window is stored there since you twice got the same hex-code - is this possible?.

Concerning the Circular List, well I don't know what macro is behind. But for the error it's again an error-window?

CoolBlue · #6 06-22-2014, 01:35 AM

Quote:

Originally Posted by whatsup

I should have said: I can imagine you can calculate the required bytes

That is if you've got a simple object and some insight on how objects are stored in memory.

Hmm. OK, that's too bad...
But anyway, the Byte Comb link you provided helps a little on that.

Quote:

Originally Posted by whatsup

The crash with Mem_Copy oTemp, lPtr, 4 when checking afterwards is still some secret to me.

OK, you don't accept my explanation in post #35?
Well, its only a theory

and I think it crashes before the line

Code:

Set ObjectFromPointer = oTemp

where I thought the problem was...
I also learned since that the first two options below will crash excel...

Code:

Sub crashMem_Copy()
Dim lptr As LongPtr
'    Mem_Copy lptr, ByVal 0, 4      'crashes excel
'    Mem_Copy lptr, ByVal lptr, 4   'crashes excel
    Mem_Copy lptr, lptr, 4            'is fine
End Sub

Quote:

Originally Posted by whatsup

Of course I tried your example with the Class (ever since I trip over your "pionterToSomething" - I should have corrected it but I didn't). What I did change, was "name as string" to "strname as string" - you know why!

Ooops... and... Ooops! two good points.

Quote:

Originally Posted by whatsup

But let's few our new "tool" - we still facing at least two problems:
1. An ordinary object requires only 4 bytes in memory. If I'm not mistaken, every set object to an object that already exists (without using the keyword "New") only is a number in the range of "As Long". But see the example it makes it clear:

Code:

Private Const coBytes As Long = 4
  
 Sub PointsToAnotherObject()
Dim objwks As Worksheet
    Debug.Print
    Debug.Print ObjPtr(Sheet1)
    pionterToSomething "Sheet1  ", ObjPtr(Sheet1), coBytes
Set objwks = Sheet1
    Debug.Print ObjPtr(objwks)
    pionterToSomething "objwks  ", ObjPtr(objwks), coBytes
    
End Sub

So in case you store ObjPtr(objwks) in a variable and check afterwards you might think the referencecount didn't work, because it's still there (unless you remove Sheet1 from your workbook). So there won't be any hint whether or not the 4 Bytes are freed, because afterwards you always will find the object of origin.

No problem, you can use VarPtr instead of ObjPtr and you will get the reference address that is stored in the local pointer. Your local variable objwks, is of course just a pointer with a reference to the worksheet object. You can see how this works here...

Code:

Sub getReference()
#If Win64 Then
    Const PTR_LENGTH As Long = 8
#Else
    Const PTR_LENGTH As Long = 4
#End If
Const coBytes As Long = 4
Dim objwks As Worksheet, lptr As LongPtr, objName As String, objwksPtr As LongPtr

    Debug.Print
'   Sheet1 Object reference and first coBytes bytes
    lptr = objPtr(Sheets(1)): objName = "Sheet1 "
    Debug.Print PointerToSomething(objName, lptr, coBytes) & vbTab & "using objPtr"
    
'   Using objPtr, can't diferentiate because the reference is resolved before reporting
    Set objwks = Sheets(1)
    lptr = objPtr(objwks): objName = "objwks "
    Debug.Print PointerToSomething(objName, lptr, coBytes) & vbTab & "using objPtr"
        
'   Using varPtr, can diferentiate.
'   The local Worksheet Object is treated as a reference, not an Object: the reference is not resolved
    lptr = VarPtr(objwks): objName = "objwks "
    Debug.Print PointerToSomething(objName, lptr, coBytes) & vbTab & "using varPtr"
    
'   The contents of the local variable (objwks) is the address of the global object but byte-reversed (little endian)
'   Load the reference into a LongPtr type and read back the byte-corrected (big endian) value of the reference
    Mem_Copy objwksPtr, ByVal VarPtr(objwks), PTR_LENGTH
    Debug.Print objName & vbTab & "points to: " & "0x" & HexPtr(objwksPtr)

'   Can also do this passing objwks as ByRef (default)
    Mem_Copy objwksPtr, objwks, PTR_LENGTH
    Debug.Print objName & vbTab & "points to: " & "0x" & HexPtr(objwksPtr)

'   Or like this...
    Mem_Copy ByVal VarPtr(objwksPtr), objwks, PTR_LENGTH
    Debug.Print objName & vbTab & "points to: " & "0x" & HexPtr(objwksPtr)
    
End Sub

Output:

Code:

Sheet1  : Address: 0x18240578 : Contents: 0xC0EA4C18    using objPtr
objwks  : Address: 0x18240578 : Contents: 0xC0EA4C18    using objPtr
objwks  : Address: 0x002AEF90 : Contents: 0x78052418    using varPtr
objwks  points to: 0x18240578
objwks  points to: 0x18240578
objwks  points to: 0x18240578

There are also some examples of how to call Mem_Copy (RtlMoveMemory) that might help to understand why it crashes.

And here you can see how the local variable's value is zero when its set to nothing...

Code:

Sub getReference2()
#If Win64 Then
    Const PTR_LENGTH As Long = 8
#Else
    Const PTR_LENGTH As Long = 4
#End If
Const coBytes As Long = 4
Dim objwks As Worksheet, lptr As LongPtr, objName As String, objwksPtr As LongPtr

    Debug.Print
'   Sheet1 Object reference and first coBytes bytes
    lptr = objPtr(Sheets(1)): objName = "Sheet1 "
    Debug.Print PointerToSomething(objName, lptr, coBytes) & vbTab & "using objPtr"
    
'   Using varPtr, can see the local variable is set to nothing.
'   The local Worksheet Object is treated as a reference, not an Object: the reference is not resolved
    lptr = VarPtr(objwks): objName = "objwks "
    Debug.Print PointerToSomething(objName, lptr, coBytes) & vbTab & "using varPtr, set to Nothing"
    
'   Using objPtr, can't diferentiate because the reference is resolved before reporting
    Set objwks = Sheets(1)
    lptr = objPtr(objwks): objName = "objwks "
    Debug.Print PointerToSomething(objName, lptr, coBytes) & vbTab & "using objPtr"
        
'   Using varPtr, can diferentiate.
'   The local Worksheet Object is treated as a reference, not an Object: the reference is not resolved
    lptr = VarPtr(objwks): objName = "objwks "
    Debug.Print PointerToSomething(objName, lptr, coBytes) & vbTab & "using varPtr"
    
'   The contents of the local variable (objwks) is the address of the global object but byte-reversed (little endian)
'   Load the reference into a LongPtr type and read back the byte-corrected (big endian) value of the reference
    Mem_Copy objwksPtr, ByVal VarPtr(objwks), PTR_LENGTH
    Debug.Print objName & vbTab & "points to: " & "0x" & HexPtr(objwksPtr)
    
End Sub

Output:

Code:

Sheet1  : Address: 0x18240578 : Contents: 0xC0EA4C18    using objPtr
objwks  : Address: 0x002AEF90 : Contents: 0x00000000    using varPtr, set to Nothing
objwks  : Address: 0x18240578 : Contents: 0xC0EA4C18    using objPtr
objwks  : Address: 0x002AEF90 : Contents: 0x78052418    using varPtr
objwks  points to: 0x18240578

Quote:

Originally Posted by whatsup

2. If memory is freed, we might find "cleared to zero", but there might be something else assigned to the pointer. Up to now it only occured when testing with pointers to a Long-variable, but I think it always can happen. Therefore we better compare - keeping in mind point 1

If the memory is set to zero or overwritten, it makes no difference; it is still clear evidence that the variable's memory space has been released. And that's been my point all along...

In summary, I cant see any reason to set local objects to nothing at the end of a sub. Even if there is an error and the sub doesn't complete normally, or if there are structures that fool the reference counting, everything is still cleaned up in the former and the set to nothing does nothing extra in the latter.

I learned a lot from this exercise and added some nice routines to my library, so this has been most interesting!

For anyone who wants to run the above routines, here is a .bas file with everything you need... well its .txt but just need to change it to .bas and you can import into a module.

whatsup · #7 06-22-2014, 06:05 PM

Sorry for the delay, but i kept thinking and thinking ...

Point1: Post #35
I basically share your opinion about the application crash on the attempt to Mem_copy something what doesn't match a 4 Bytes Code.
By accident I choose first a range-object (and experienced the crashes), afterwards I changed to a worksheet-object (avoiding crashes I didn't any mem-copy to an object or a variant). I was just looking at the numbers of ObjPtr which are always the same whether you obtain them directly from the object
ObjPtr(Sheet1)
or from the variable
ObjPtr(objwks)
Therefore I wondered, why did excel crash, because even with the variable gone I ask at the end of the Copy of ObjPtr(Range("A1")), and since the Range still exists and hasn't changed, what's the problem to copy it?
It took me a long time to figure out there is a difference between Range and Worksheet:
Whereas Worksheet keeps his pointer throughout the application, this isn't the case with Range. A Range changes its pointer, furthermore the variable (objRange) set to the Range is assigned another pointer.

I said the choice was by accident but actually I'm glad about the choice because otherwise it would have led in a complete different direction...

Point2:
I regret the lack of knowlegde about the architecture of objects and what they look like in memory. I really do, probably it would make things a lot easier knowing about this things.

Point3:

Quote:

If the memory is set to zero or overwritten, it makes no difference; it is still clear evidence that the variable's memory space has been released. And that's been my point all along...

I would feel more comfortable if evidence would present itself more obvious, for example in just showing only zeros (as it does with a simple class). Why can't excel give a pointer some rest after he did a job

But Yes, I agree, at least for the moment let's depend on the change.

Point4:
According to Point 4, I agree, memory is freed whether or not the variable is set to Nothing (at least valid for the macros I tried up to now). A surprise to me: Classes are destroyed without explicity destroying them.
But I don't agree on this

Quote:

...or if there are structures that fool the reference counting...

If you refer to the example with circular references within classes, that's one thing which the example establishes most clearly: None of them get destroyed.

Attached the file including 4 tests - as well the CircRef - with some kind of summary.

CoolBlue · #8 06-22-2014, 07:37 PM

Quote:

Originally Posted by whatsup

Sorry for the delay, but i kept thinking and thinking ...

Point1: Post #35
I basically share your opinion about the application crash on the attempt to Mem_copy something what doesn't match a 4 Bytes Code.
By accident I choose first a range-object (and experienced the crashes), afterwards I changed to a worksheet-object (avoiding crashes I didn't any mem-copy to an object or a variant). I was just looking at the numbers of ObjPtr which are always the same whether you obtain them directly from the object
ObjPtr(Sheet1)
or from the variable
ObjPtr(objwks)
Therefore I wondered, why did excel crash, because even with the variable gone I ask at the end of the Copy of ObjPtr(Range("A1")), and since the Range still exists and hasn't changed, what's the problem to copy it?
It took me a long time to figure out there is a difference between Range and Worksheet:
Whereas Worksheet keeps his pointer throughout the application, this isn't the case with Range. A Range changes its pointer, furthermore the variable (objRange) set to the Range is assigned another pointer.

I said the choice was by accident but actually I'm glad about the choice because otherwise it would have led in a complete different direction...

I'll have to take some time to think about that, but they are interesting observations...
I'm surprised that the crashMem_Copy routine in my previous post crashes excel because the destination pointer is fine. Why crash when the source pointer is set to zero? Its only a read... Any thoughts?

Quote:

Originally Posted by whatsup

I regret the lack of knowlegde about the architecture of objects and what they look like in memory. I really do, probably it would make things a lot easier knowing about this things.

Well, clearly you are not alone! Nobody seems to have any knowledge about this, at least I cant find any.
But your link was very helpful, I've seen the byte comb site before but it was good to be reminded of it, it informed my thinking on this.

Quote:

Originally Posted by whatsup

I would feel more comfortable if evidence would present itself more obvious, for example in just showing only zeros (as it does with a simple class). Why can't excel give a pointer some rest after he did a job

But Yes, I agree, at least for the moment let's depend on the change.

It seems to me that the Erase Method in VBS's "garbage collection" system does indeed set the first 8 bytes to zeros. The point is, after a crash and re-start of another routine, a LOT has changed and the space will surely have been used for something else. So the zeros have been over-written.

Quote:

Originally Posted by whatsup

According to Point 4, I agree, memory is freed whether or not the variable is set to Nothing (at least valid for the macros I tried up to now). A surprise to me: Classes are destroyed without explicity destroying them.

Yes, I saw from one of your early quotes that you would be surprised about the classes being destroyed and I wanted to give you clear evidence about that and that's why I did the class example.
Its quite simple really, all object life cycle is managed by reference counting: when the reference count goes to zero, the object is scheduled for deletion. There are no no exceptions.
I guess this will be a background process that is run when VBARuntime has free time and it will just scan the reference count field in the object table and "Erase" those with zero reference. The ERASE method will not delete the object from the object table, but it will release any memory used by the object's structure.

Quote:

Originally Posted by whatsup

But I don't agree on this
If you refer to the example with circular references within classes, that's one thing which the example establishes most clearly: None of them get destroyed.

Attached the file including 4 tests - as well the CircRef - with some kind of summary.

OK, now we get to the main point

The full quote on this is:

Quote:

Originally Posted by CoolBlue

In summary, I cant see any reason to set local objects to nothing at the end of a sub. Even if there is an error and the sub doesn't complete normally, or if there are structures that fool the reference counting, everything is still cleaned up in the former and the set to nothing does nothing extra in the latter.

Sorry, the language is a bit unclear so I've added emphasis...

I am saying that setting to nothing does nothing extra in these cases. Im saying it makes no difference. The problem will still be there.
I am arguing against using set to nothing remember?

As you can see in your example, the result after run-time is the same if you have

Code:

Set objcls1 = Nothing

Or

Code:

'Set objcls1 = Nothing

THAT is my point: you cannot fix these issues with set to nothing. One of the links offered by my friend @macropd as a reason to use set to nothing was referring to these structures and my point was and still is that set to nothing makes no difference.

So, the two points I was trying to argue against are:

Your second point in post#7

Quote:

Originally Posted by whatsup

- The main reason: There are different types of objects, for instance classes you must unload manually; if creating a new instance of an object (whenever using the keyword "New") you have to unload it manually as well...

The claim by @macropod in post#7 that this link was supporting the use of set to nothing at the end of a sub as a way to manage (for example) objects with circular references.

I think we do agree on those...

And thanks for the attachment... your documentation style is much nicer than mine, I will study it and improve my ways!

CoolBlue · #9 06-20-2014, 06:45 PM

Here are the results for a circular list compared to a single object...

Single Object

Code:

No Cleanup, No Error
SingleObject set...
objCls : 0x18DECA08 : 0x64C67D1C0000000024CADE18EC577803D45778030000000000000000ECECAE1B0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC00000000002489C3180000000000000000FD3D013B6500008C0000000028000000
after exit...
objCls : 0x18DECA08 : 0x000000000000000024CADE1800000000000000000000000000000000ECECAE1B0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC00000000002489C3180000000000000000FD3D013B6500008C0000000028000000

With Cleanup, No Error
SingleObject set...
objCls : 0x18DECA08 : 0x64C67D1C0000000024CADE18EC577803D45778030000000000000000ECECAE1B0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC00000000002489C3180000000000000000FD3D013B6500008C0000000028000000
Set to Nothing...
objCls : 0x18DECA08 : 0x000000000000000024CADE1800000000000000000000000000000000ECECAE1B0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC00000000002489C3180000000000000000FD3D013B6500008C0000000028000000
after exit...
objCls : 0x18DECA08 : 0x000000000000000024CADE1800000000000000000000000000000000ECECAE1B0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC00000000002489C3180000000000000000FD3D013B6500008C0000000028000000

With Cleanup, Error after setting the pointer
SingleObject set...
objCls : 0x18DECA08 : 0x64C67D1C0000000024CADE18EC577803D45778030000000000000000ECECAE1B0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC00000000002489C3180000000000000000FD3D013B6500008C0000000028000000
Set to Nothing...
objCls : 0x18DECA08 : 0x000000000000000024CADE1800000000000000000000000000000000ECECAE1B0100000000000000000000006E1C000000000000AAAAAA00ABABAB00BCBCBC00000000002489C3180000000000000000FD3D013B6500008C0000000028000000
After Error, Before End Sub
after error...
objCls : 0x18DECA08 : 0x8841BF182AA30000712CBB1BDC25BB1BC6D3B71B10000400F88656080000000000000000000000000000000006000000100000005029881C000000005F005F005300520050005F000000000000000000FD3D013B6500008C0000000028000000

No Cleanup, Error after setting the pointer
SingleObject set...
objCls : 0x18DECA08 : 0x64C67D1C0000000024CADE18B45778039C5778030000000000000000ECECAE1B0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC00000000002489C3180000000000000000FD3D013B6500008C0000000028000000
After Error, Before End Sub
after error...
objCls : 0x18DECA08 : 0x8841BF182AA30000712CBB1BDC25BB1BC6D3B71B1000040030865608000000000000000000000000000000000600000010000000D82B881C000000005F005F005300520050005F000000000000000000FD3D013B6500008C0000000028000000

Circular List

Code:

No Cleanup, No Error
circularList set...
objCls : 0x18DECA08 : 0x64C67D1C0000000024CADE18EC577803D45778030000000000000000ECECAE1B0200000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC00C8CC82182489C3180000000000000000FD3D013B6500008C0000000028000000
after exit...
objCls : 0x18DECA08 : 0x64C67D1C0000000024CADE18EC577803D45778030000000000000000ECECAE1B0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC00C8CC82182489C3180000000000000000FD3D013B6500008C0000000028000000

With Cleanup, No Error
circularList set...
objCls : 0x0DA0DC70 : 0x64C67D1CC8CC82188CDCA00D245878030C5878030000000000000000ECECAE1B0200000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC0000A9B70D2489C31800000000000000005E55B53900000088E073AF1B08179D0D
Set to Nothing...
objCls : 0x0DA0DC70 : 0x64C67D1CC8CC82188CDCA00D245878030C5878030000000000000000ECECAE1B0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC0000A9B70D2489C31800000000000000005E55B53900000088E073AF1B08179D0D
after exit...
objCls : 0x0DA0DC70 : 0x64C67D1CC8CC82188CDCA00D245878030C5878030000000000000000ECECAE1B0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC0000A9B70D2489C31800000000000000005E55B53900000088E073AF1B08179D0D

With Cleanup, Error after setting the pointer
circularList set...
objCls : 0x1882CDD0 : 0x64C67D1C00A9B70DECCD8218945878037C5878030000000000000000ECECAE1B0200000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC0078CD82182489C318000000000000000042AC103B000000804400FA171A000000
Set to Nothing...
objCls : 0x1882CDD0 : 0x64C67D1C00A9B70DECCD8218945878037C5878030000000000000000ECECAE1B0100000000000000000000000F10000000000000AAAAAA00ABABAB00BCBCBC0078CD82182489C318000000000000000042AC103B000000804400FA171A000000
After Error, Before End Sub
after error...
objCls : 0x1882CDD0 : 0x0D00E7719425E7717825E7715825E7714825E771000000006890851868908518000000000000000078908518000000000900000000000000000000000000000001000000DCCD821800000000FA050E0042AC103B000000802E00E7719425E771

I think we've found the higgs boson!
The detector works...

You have to scroll across a bit but you can see that the object is not released when there is a circular reference. Its not released by VBA or by the set to nothing.

CoolBlue · #10 06-23-2014, 06:35 AM

I am attaching a better version for handling the endian-ness...
This version is actually endian-agnostic: I let VBA figure it out and we always get the correct answer.
The routine is called Mem_ReadHex_Words. It also allows for 4, 8 or 16 bit words which are required by some data structures.
I use the idea of loading the memory image, one word at a time, into a LongPtr Type and then convert the LongPtr to hex. All the underlying byte swapping is taken care of by VBA automatically.

whatsup · #11 06-23-2014, 09:00 AM

Quote:

Ah, this one I can explain...

Ey, that's excellent news. Now that you mention it I remember reading about it on the byte comb page, but didn't bother pay attention to it

I will have a look at your files later on, is to say in the evening, still got some other work to do ...

whatsup · #12 06-23-2014, 04:54 PM

O_o, quite interesting

May I ask you a few questions about your new function of the last upload?
1.

Code:

Public Const coBytes As Long = 4 * 8

You changed to 32 Bytes, just for purpose of demonstration or is there any particular reason?
2.

Quote:

The routine is called Mem_ReadHex_Words. It also allows for 4, 8 or 16 bit words which are required by some data structures.

That's still the problem, that we don't know what data-structure we're really facing, isn't it?
3.
Curiousity - I've never seen this before:

Code:

wordCount = Length \ wordLength

For divisions I use / you as well as I see in the following line. Therefore I wonder what's the joke on this?

CoolBlue · #13 06-23-2014, 07:41 PM

Quote:

Originally Posted by whatsup

O_o, quite interesting

May I ask you a few questions about your new function of the last upload?
1.

Code:

Public Const coBytes As Long = 4 * 8

You changed to 32 Bytes, just for purpose of demonstration or is there any particular reason?

The main reason is that the object structures are generaly more than one worrd so I wanted to see what was there in the structure to see if I could notice any patterns now that the byte order is fixed.
The Mem_Read for VarPtr is still set to 4 (it should be set to POINTER_LENGTH in fact).

Quote:

Originally Posted by whatsup

2.
That's still the problem, that we don't know what data-structure we're really facing, isn't it?

Well, the data structures are interesting but not relevant to the test we are doing. But anyway, we now have a method for seeing the data properly (not scrambled by endian coding) so we can look at some standard structures like VB Variant structure for example to see if we can get some clue.
The main point is that we can confirm the relationship between VarPtr and ObjPtr for objects. We can confirm that the Loal Object, is just a LongPtr containing the address of the referenced object or zero, depending on if its been set or not.
Before I unscrambled the bytes, it was not obvious that the Mem_Read from VarPtr was in fact the address of the global object. But now we are clear on that.
So, using your Mod01 as an example, it means that we can say with 100% confidence that, after runtime:
If Mem_ReadHex_Words(lng_objPtr, 4) = 0 then
objwks is set to Nothing

If Mem_ReadHex_Words(lng_objPtr, 4) = ObjPtr(Sheet1) then
objwks has not been properly released

If Mem_ReadHex_Words(lng_objPtr, 4) <> ObjPtr(Sheet1) then
objwks has been properly released

Thus solving the problem that you raised earlier about ObjPtr returning the same thing for the local and global variables.

Quote:

Originally Posted by whatsup

3.
Curiousity - I've never seen this before:

Code:

wordCount = Length \ wordLength

For divisions I use / you as well as I see in the following line. Therefore I wonder what's the joke on this?

/ is divide and \ is div or integer divide
Run this code...

Code:

Sub testIntDiv()
Debug.Print 3 / 5
Debug.Print 3 \ 5
End Sub

I use it like this:
if p / q = p\ q then p is divisible by q

whatsup · #14 06-24-2014, 06:03 PM

Thanks, for the explanations

I agree and I'm delighted that your function Mem_ReadHex_Words shows the relation to HexPtr(ObjPtr) clearly now.
On the proof for the release of memory I also agree, though it's more complicated as you showed:

Quote:

If Mem_ReadHex_Words(lng_objPtr, 4) = ObjPtr(Sheet1) then
objwks has not been properly released

If Mem_ReadHex_Words(lng_objPtr, 4) ObjPtr(Sheet1) then
objwks has been properly released

Actually:
If Mem_ReadHex_Words(lng_varPtr, 4) doesn't point anymore to
HexPtr(lng_objPtr, 4)
And
Mem_ReadHex_Words(lng_objPtr, 4) <> Mem_ReadHex_Words(ObjPtr(Sheet1))
should proof that memory has been released. (But that's just for the record, I think you meant the same)

Quote:

The Mem_Read for VarPtr is still set to 4 (it should be set to POINTER_LENGTH in fact).

Yes, I was setting it to 4 knowing the value (for objects) has to be in the range of Long. I didn't intend to test anything else with the tool.

Therefore, all we have to do: Thinking up a routine where reference count fails

CoolBlue · #15 06-24-2014, 07:42 PM

Quote:

Originally Posted by whatsup

On the proof for the release of memory I also agree, though it's more complicated as you showed:
Actually:
If Mem_ReadHex_Words(lng_varPtr, 4) doesn't point anymore to
HexPtr(lng_objPtr, 4)
And
Mem_ReadHex_Words(lng_objPtr, 4) Mem_ReadHex_Words(ObjPtr(Sheet1))
should proof that memory has been released. (But that's just for the record, I think you meant the same)

Yep, thanks for pointing that out, I left out the in my post and have now corrected it.

Quote:

Originally Posted by whatsup

Yes, I was setting it to 4 knowing the value (for objects) has to be in the range of Long. I didn't intend to test anything else with the tool.

Actually I thought it was me who changed your code and left the 4 there

Im building a class module now to encapsulate this stuff to make it more convenient to deploy. I have some problem related to the need to pass the variable/object into the class as a variant type, i need to make sure the original address is available inside the class, so I just need solve that. I guess its got something to do with decoding the Variant Type structure but the Bytecomb site is very helpful on that, so shouldn't be a problem i think...

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Wierd "script code" in a downloaded .doc file	CNBarnes	Word	2	10-18-2012 02:07 AM
replace data from variable with "sub and super script" from excel to word by vba	krishnaoptif	Word VBA	9	06-22-2012 05:08 AM
How to choose a "List" for certain "Heading" from "Modify" tool?	Jamal NUMAN	Word	2	07-03-2011 03:11 AM
Rules and Alerts: "run a script"?	discountvc	Outlook	0	06-15-2010 07:36 AM
An "error has occurred in the script on this page"	decann	Outlook	8	09-03-2009 08:54 AM