Unfortunately it is pretty much all or nothing. You have a pretty good method of keeping those addresses that you want while getting rid of the rest. Unfortunately there's no way to configure Outlook to ask for confirmation before adding an address to the Auto-complete list.

One thing to keep in mind is that the Auto-complete list was never meant to be a permanent repository of addresses. Basically it's there for convenience sake, but addresses that are emailed frequently or that you want to keep long term really should be kept as a Contact. The auto-complete list is highly susceptible to becoming corrupt or storing invalid addresses, and I recommend clearing the list on a regular basis.

-SW