I think that the "X" and "Y" in "Page X of Y" are fields. You can see fields in the document by typing Alt-F9. (Type Alt-F9 again to hide them.)
You can delete fields in the headers with a for loop.
Code:
Sub delete_fields_in_header()
Dim x As Integer, num_fields As Integer
num_fields = ActiveDocument.Sections(1).Headers(wdHeaderFooterPrimary).Range.Fields.Count
For x = num_fields To 1 Step -1
ActiveDocument.Sections(1).Headers(wdHeaderFooterPrimary).Range.Fields(x).Delete
Next x
End Sub