The button looks like a end-of-paragraph marker and is located on the Home tab. I think in the Paragraph group.

This tool is extremely valuable. Consider learning to leave it on much of the time while you are creating a document.

Working in Word without toggling on and off the display of the "non-prints", is like tying one hand behind your back.