Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MODEL] Added functionality to provide custom value of total count for pagination #1023

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

bk-az
Copy link

@bk-az bk-az commented Mar 20, 2022

The Problem

When using collapse parameter to collapse search results, the total number of hits in the response indicates the number of matching documents without collapsing. As a result, the total number of pages for pagination is miscalculated.

Assume, for example, the following query on the group_items index returns a total of 100 results without collapsing.

GET group_items/_search
{
  "query": {
    "match_all": {}
  }
}

And when the search results are collapsed based on the group_id field value. The total results returned by the query becomes only 5 instead of 100.

GET group_items/_search
{
  "query": {
    "match_all": {}
  },
  "collapse": {
    "field": "group_id"         
  }
}

The problem occurs when we paginate results, after collapsing, the value of the total number of hits is still 100 in response, which results in the miscalculation of the total number of pages, as mentioned in the reference:

The total number of hits in the response indicates the number of matching documents without collapsing. The total number of distinct groups is unknown.

The solution

This problem with pagination can be solved by providing a custom value of total_entries (will_paginate) or total_count (kaminari). The value of total distinct groups can be calculated by using cardinality aggregation

# Assuming a method `total_collapsed` is defined to get the total count of 
# collapsed results, using cardinality aggregation.
total_count = total_collapsed(response.aggregations)

Now to get pagination working correctly, the value of total_count can be specified.

# for will_paginate
response.paginate(page: 1, per_page: 30, total_entries: total_count)

# for kaminari
response.page(1).total(total_count)

@bk-az
Copy link
Author

bk-az commented Mar 29, 2022

@picandocodigo can u please review this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant