The solution for me:

I changed the minimum number on the Value Axis to 0.

When I let Excel create the default chart, it started the axis at 80% which is why it makes the stacked bar chart look incorrect.