The important thing is that you need a section break each time you want the page number to restart. You need to use the built-in headings for page numbers to pick up the heading numbers. To instruct Word to pick up the heading number, make use of the Page Number Format dialog box (access that dialog box once per section).

The functionality has been the same in Word for many years, and it is covered in Shauna Kelly's article.

This question has also been posted in the TechCommunity at Insert page number in table of content - Microsoft Tech Community where I have uploaded a sample document. The sample document is very simple in the sense that there are only two sections: one for the body text and one for the appendix. If you want more page number restarts, you'll need more sections.

__________________
Stefan Blom
Microsoft Word MVP

Microsoft 365 apps for business
Windows 11 Professional