Because of :-\?, your Find expression say, amongst other things, to find all characters between : and ?. That includes :, ;, <, =, >, and ?. You need to use \-, as in:
.Text = "[.,;:\-\?\!^13^34]"
And, yes, you can use ^n, where n is a number, to look for ASCII characters - but for the most part you can also use the characters themselves.

__________________
Cheers,
Paul Edstein
[Fmr MS MVP - Word]