To delete it: you need to select the line clicking beside it then use Del, or, use Undo just after it happens.
To avoid it happening: go to menu File / Options / Proofing / Autocorrect Options... / and in tab "Autoformat As You Type" / unselect "Border Lines" in section "Apply As You Type".

__________________
Difficult is not to know but to share what you know (Han Fei Tzu reworked)