Every section will have three different headers and three different footers even if you never see them. Because of this, and because there can be multiple sections on a single page, it may be easiest to temporarily expand each section to at least three pages (to see the First-Page, Even-Page, and Odd-Page headers and footers. This way you can see what is in each header/footer and what the settings are. You can temporarily expand the number of pages using manual page breaks (Ctrl+Enter).

Header/Footer Settings Recap

In your document you would want your sections that start a new chapter set to have a different first-page header. That one can be empty. You will then be using all three headers. If you use a StyleRef Field you can have all of your section headers (after the first) linked to the previous section. The StyleRef field can be used to pick up your chapter title.