Thanks for your reply Stefan. Your suggestion isn't what I wanted to accomplish (What I want is for the pages to appear like all the Chapter 2 pages here: http://i.imgur.com/GLqbzzk.png), but after playing with the page number format page, I realized that that window only affects the "active" chapter you have selected. I thought that window, if you changed it, affected the entire document, but it only affects your "current" chapter selected. This is why when I was changing it earlier (I probably had chapter 1 or 2 active at the time), that window was not producing the results I wanted.

Basically, the culprit was "Chapter starts with Style:" option. Once I modified that on chapter 3, it was fine.