If you want the conditional formatting to work with Y/N then you need to use:

Home (tab) > Conditional Formatting > Manage rules > New Rule > Format only cells that contain > [Cell Value - equal to - Y] > Format [choose whatever you want]
Follow the steps from 'New Rule' again to do the other Conditional Format.

I think what's happened is that you used the 'Text that contains ...' option - meaning that ANY word with Y/N in will activate the rule! Ooops!