Easy. Type Alt+S,P (Alt + S then P) to open the Caption dialogue. To the default numbering, simply add the chapter number by checking the Include chapter number box, and use the dropdown of Separator to select the period.
Done.