Originally Posted by ArviLaanemets

A simplest solution - replace all non-date numeric entries (year numbers) with dates. E.g. instead 1965 enter a date like 01.01.1965, and instead 1985 enter 01.01.1985.
In case you want to display them as year numbers, simply format those cells as custom "yyyy" (but this is not obligatory). Now sorting by date returns exactly what you want.

In case those data are read dynamically form some database, the solution can be more tricky, involving a helper column where all non-date entries are replaced with calculated dates, and date entries with copy of dates. Then you can sort by this helper column.