Tables consist of rows, columns and cells, not boxes, but I think you are wanting to set the height of rows.

Right-click on the Table and choose Table Properties. Go to the Row tab. When you specify height, be sure to also set the "Row height is" drop-down to Exactly.

This sets it for whatever row you right-clicked on. Notice there are Previous Row and Next Row buttons.