Skip to content

Search challenges - cannot page through more than 10000 challenges #502

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Jaouad-Jaghrir opened this issue Jul 14, 2022 · 0 comments
Open
Labels
bug Something isn't working

Comments

@Jaouad-Jaghrir
Copy link

When we perform a search for challenges with page=100 and perPage=100, I see that the number of total pages is 583
See below:
search-page100

But when I try to search with page=101 and perPage=100, there are no results returned by the API and the values of total-pages and total number of records are set to 0 which is wrong:
wrong-total-pages

Root cause:

The issue here is due to the pagination parameters used to get the data from Elasticsearch es-query-pagination

According to ES documentation By default, you cannot use from and size to page through more than 10,000 hits. This limit is a safeguard set by the [index.max_result_window](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#index-max-result-window) index setting. If you need to page through more than 10,000 hits, use the [search_after](https://www.elastic.co/guide/en/elasticsearch/reference/current/paginate-search-results.html#search-after) parameter instead.

See es-paginate-search-result

cc @ThomasKranitsas

@Jaouad-Jaghrir Jaouad-Jaghrir added the bug Something isn't working label Jul 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant