Thanks for your reply Lugh,

What happens is that if I open something like Word, when I go back to Outlook it's still open but no emails are shown.

I've looked at your suggestion:

File > Options > Advanced > Outlook start and exit

I highlighted the folder that I wanted Outlook to open on and clicked OK. It worked! A simple solution that I didn't know existed. How it got that way in the first place, I don't know!