With Word closed - Copy the path in blue below:

%Userprofile%\AppData\Roaming\Microsoft\Templates

. . . . then open a File Explorer Window, select the Address Bar, then r/click in the Bar and paste what was copied into the Bar.
Now click the Arrow tab at the end of the Address bar, or press the Enter key.
Locate the Normal(.dotm) file in the opening Window, then r/click it > Open.

Word will open with the Normal template.
Change the orientation to Portrait, then Save.
Close Word, close the Templates folder, then re-open Word and the opening document should be portrait and all new documents in the future should be.