You have found the most useful tool in all of Microsoft Word. Those symbols are called "non-printing characters", or commonly called the "non-prints". Experienced Word users toggle them on and off frequently, or just leave them on. Without using them, you are flying blind, unable to see when the tab key was pressed, when spaces were inserted, manual page breaks don't show up, and of course, the end-of-paragraph marker (aka pilcrow), is not shown.

In 2003, the button to toggle on and off the display of non-prints is on the Standard toolbar. It looks like a pilcrow and is called Show/Hide.