I do not see how you have much choice. The behaviour describe is the behaviour of non-table (i.e. "newspaper") columns. Newspaper columns start top left and move to right (assuming two-column setup) per page. What you want it seems is one column extending the whole document. No matter how many pages.

A table. As for setting up, there does not seem much in the way of setting up. Make a new document. Make a two column table say seven inches wide. Make the second column 1 inch. Copy all the content (Ctrl-A) of the document you want in the new document. Paste it into the first column.

Clearly there may be other issues come up, but as you have stated it, it does not seem hard, or difficult to set up. Not that many documents look all that good in a six column width, but...

Last comment: newer versions of Word may allow newspaper columns stretching over an entire document. I have not heard of such though.