Try starting Outlook once with the /cleanreminders command switch.
Start>Run>outlook.exe /cleanreminders
Be sure to include the space.

This will reset the reminders and should allow them to properly display.