You cannot using the Columns feature which is Section formatting. These are intended to be newspaper or magazine style columns.

You can have a two-column Table and it will work the way you want. You will not need a section break. You can turn borders off as you wish.