That is a feature of Print Layout view in Word 2013. Every paragraph that has an Outline Level other than Body Text (mostly those with styles of Heading 1, Heading 2, etc.) will show that arrow in the left margin when the mouse cursor hovers over the paragraph's text. Clicking it collapses everything below the heading down to the next heading of the same level. When the heading is collapsed, clicking the arrow again expands the text. The functionality is a subset of what you get in Outline view.

There is no way to get rid of it unless you remove Office 2013 and go back to an earlier version.