Apparently, some Avery Templates have an outline/Shape around the label.
This is easily removed by selecting it and pressing the Delete key, d/clicking it > Shape Outline (Drawing Tools/Format) > No Outline, or r/clicking it > Format Shape > Line Color > No Line > Close.