Elastic is currently limited do only allow searches up to 10 000 results, probably by the default setting of max_result_window = 10000. This means that it is not possible to load pages 417 and after with a page size of 24.
Categories with more than 10 000 products can happen on the root category on sites with many products. Right now it needs to be solved by limiting the number of pages that can be displayed, in some cases the preferred solution would be to be able to actually display all pages.