No, there is no way to delete those two rows as long as they are not adjacent.

One thing you can do is drag one of the rows and drop it immediately above or below the other row (or drag row 3, and drop it above row 2 or below row 4). Once the two rows are adjacent, you can select and delete them together.

However, it's generally quicker just to delete first one of the rows and then the other.