Note, this will not work if there is an active search.
While in an active search, you can tab to the x close button, though, to close the search using the space bar. Then the toggle work