Identify duplicate phrases within a document

[bianson]

I have a multi page Word document where I have typed in quotations I have collected over the years. The problem is that I have hundreds and hundreds of these quotations and now realize that some have been duplicated over the years. Is there a process or a software package that would scan my document and identify the duplicates which I could then safely remove?

[macropod]

What kind of formatting is applied to the quotations? Do they cite authors and, if so, how are the author names separated from the quotations to which they refer?

[bianson]

Just the quote and the author but both would be duplicated. I am not sure if I answered your question properly?

[macropod]

What you haven't said is how the content is formatted and how the author names are separated from the quotations to which they refer (e.g. are they in the same paragraph, or is there a paragraph break between them).

[Guessed]

Probably not. There is a good chance that there are minor differences which make many 'duplicates' not exactly the same.

But you could arrange the data in a way so that it could be sorted alphabetically and then you could either eyeball the duplicates or run a macro to highlight exact matches.

It all depends on what your document is structured like and how consistent that structure is.

[macropod]

One could probably copy/paste the content into Excel, then sort it there. Any exact or near duplicates are likely to be sorted together. One could then locate & delete the corresponding duplicates in Word via Find.