Select both and group them. Grouping should be in a right-click menu as well as in the contextual Ribbon tab. It may be under "Arrange."

If the photo and caption are "in-line-with-text" then have the paragraph formatting of the paragraph containing the photo set for keep with next if the caption is below the photo. Moving a Figure and Its Caption by Charles Kenyon