Look at the horizontal ruler. Use the tab settings. Usually in a header there is a center tab at the center and a right tab at the right margin. You can remove the center tab from the Header style if you don't use it.

Remember, when trying to figure out what is doing things in a Word document, always turn on display of nonprinting formatting characters.