I finally fixed it. In case anybody runs into the same issue: Just click on the user on top of office. There you will see several people logged into Office. Log out the ones you don't want to see. And that's it.