Skip to content

fix: Connection pagination refactor#2408

Closed
justlevine wants to merge 30 commits intowp-graphql:developfrom
justlevine:fix/connection-pagination
Closed

fix: Connection pagination refactor#2408
justlevine wants to merge 30 commits intowp-graphql:developfrom
justlevine:fix/connection-pagination

Conversation

@justlevine
Copy link
Copy Markdown
Collaborator

@justlevine justlevine commented Jun 18, 2022

What does this implement/fix? Explain your changes.

This PR aims to handle a slew of issues with paginating on GraphQL connections.
Originally surfaced by #2343, a review showed not just regressions caused by #2294, but others dating back to v1.6.x as well.

  • So far, connection pagination tests have been refactored and fleshed out to expose the bugs.
  • Instead of letting the tests fail, a markTestIncomplete() notes where the first bug is surfaced. As those are addressed, it's possible the remaining test code will need to be tweaked.

Next step is using those tests to fix the exposed issues. Work will continue on this draft PR.

Contributions are welcome, either as comments on this PR, or by your own commits - just remember to git rebase justlevine/fix/connection-pagination, resolve any conflicts, and git push justlevine/fix/connection-pagination --force when you're done to keep the commit history clean.

Does this close any currently open issues?

Will be filled out when the PR is read for merge.

Any relevant logs, error output, GraphiQL screenshots, etc?

(If it’s long, please paste to https://ghostbin.com/ and insert the link here.)

Any other comments?

...

Where has this been tested?

Operating System: Ubuntu 20.04 (wsl2 + devilbox + php 8.0.15)

WordPress Version: 6.0

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 6806 lines exceeds the maximum allowed for the inline comments feature.

@justlevine justlevine added type: bug Issue that causes incorrect or unexpected behavior scope: tests Developing unit tests, integration tests, and ensuring coverage effort: high More than a week component: pagination Relating to pagination regression Bug that causes a regression to a previously working feature labels Jun 18, 2022
Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 6813 lines exceeds the maximum allowed for the inline comments feature.

@coveralls
Copy link
Copy Markdown

coveralls commented Jun 18, 2022

Coverage Status

Coverage increased (+0.1%) to 80.676% when pulling 7e99bed on justlevine:fix/connection-pagination into 16b159f on wp-graphql:develop.

@justlevine
Copy link
Copy Markdown
Collaborator Author

First bug:

Fetching multiple comments in one query (or in one Codeception test instance), recycles the original ids instead of fetching new ones.

To replicate:

  1. Create and approve 3 comments.
  2. Run the following query
{
  comments(first:2) {
    pageInfo {
      startCursor
      endCursor
    }
    edges {
      cursor
      node {
        databaseId
      }
    }
  }
  after:comments(first:1, after:"YXJyYXljb25uZWN0aW9uOjM="){ # the second cursor.
    pageInfo {
      startCursor
      endCursor
    }
    edges {
      cursor
      node {
        databaseId
      }
    }
  }
}

All results should be shown correctly. The reason for this is that the first comments query overfetches by one.

  1. Change the comments(first:2) to comments(first:1) and rerun the query.
    You should see after returns null. This is because the comment query overfetch only includes the first two.

Screenshots

  • first:2
    image
  • first:1
    image

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 6819 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 6873 lines exceeds the maximum allowed for the inline comments feature.

@jasonbahl
Copy link
Copy Markdown
Collaborator

@justlevine I see that this PR removed a lot of tests.

For example:

  • tests/wpunit/CursorPaginationForPostsTest.php
  • tests/wpunit/CursorPaginationForCategoriesTest.php

etc.

Were these removed because they're duplicate tests and have coverage elsewhere? or was there another reason for removing them?

);
$second_to_last_comment = $comments[0];
codecept_debug( $expected );
$this->markTestIncomplete( 'works until here' );
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justlevine For this to work, we'll need to add functionality similar to what's been done for Post and Term connections:

See: https://github.com/wp-graphql/wp-graphql/blob/develop/src/Data/Config.php#L187-L216

Copy link
Copy Markdown
Collaborator

@jasonbahl jasonbahl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justlevine I like what you're doing here for sure.

There's a lot of rough patches with pagination and it would be lovely to get them all shored up.

I anticipate this taking some time to do comprehensively, so I propose we at least get a fix for the taxonomy pagination regression first, then continue working through other pagination issues as a focus of another release.

I have a branch locally where I have the Taxonomy pagination working that I can PR up with a few tests, so that we can unblock the Gatsby users that are suffering from the regression, then we can keep working on this to get things in a better spot.

@justlevine
Copy link
Copy Markdown
Collaborator Author

Were these removed because they're duplicate tests and have coverage elsewhere? or was there another reason for removing them?

Mostly duplicates. What wasnt has been relocated to the {ConnectionType}QueriesTest/{ConnectionType}PaginationTest to keep a standard pattern and ensure tests arent duped/overlooked in the future.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 7538 lines exceeds the maximum allowed for the inline comments feature.

@justlevine
Copy link
Copy Markdown
Collaborator Author

justlevine commented Jun 21, 2022

Okay first pass is done:

Refactored AbstractConnectionResolver to better follow the Connection spec, while using smaller methods to allow easier creation of child classes. Of note.

  • get_offset() is deprecated in favor of get_offset_for_cursor( string $cursor ), so we can support before + after together.
  • get_offset_for_cursor() works with either a string|int offset, and is passed to get_array_index_for_offset( $offset, array $ids) to handle where in the array we slice.
  • The big change is the lifecycle of when we're slicing/reversing our IDs:
    • get_ids() is no longer abstract and now runs get_ids_from_query() and apply_cursors_to_ids( array $ids ). The former gets the IDs from $this->query and sets them in the correct order; the latter handles the new overfetched slicing logic.
      For b/c get_ids_from_query() is not abstract. Instead an Exception is thrown if the class doesnt either implement that method or get_ids().
    • get_ids_for_nodes() slices (the overfetched) $this->ids to the actual connection query size.
  • is_valid_offset() has been added to missing classes.

To Do

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 7554 lines exceeds the maximum allowed for the inline comments feature.

@justlevine
Copy link
Copy Markdown
Collaborator Author

Those cests should not be failing 🤔

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 7559 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 7559 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 7559 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 12040 lines exceeds the maximum allowed for the inline comments feature.

@justlevine
Copy link
Copy Markdown
Collaborator Author

Turns out most tests don't clean up after themselves, and some use duplicate names for objects causing tests to pass that shouldnt.

Tried to catch/fix as many as I could. I think all thats left are WP_Post and WP_User objects.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 12209 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 12305 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 12310 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 12516 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 12535 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 12736 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 13148 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 14078 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 14048 lines exceeds the maximum allowed for the inline comments feature.

Copy link
Copy Markdown

@qlty-cloud-legacy qlty-cloud-legacy bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR diff size of 15459 lines exceeds the maximum allowed for the inline comments feature.

@qlty-cloud-legacy
Copy link
Copy Markdown

Code Climate has analyzed commit c7d9c9b and detected 3 issues on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 1
Duplication 2

View more on Code Climate.

@justlevine
Copy link
Copy Markdown
Collaborator Author

Work moved to #2457

@justlevine justlevine deleted the fix/connection-pagination branch February 10, 2024 19:38
@justlevine justlevine restored the fix/connection-pagination branch February 10, 2024 19:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component: pagination Relating to pagination effort: high More than a week regression Bug that causes a regression to a previously working feature scope: tests Developing unit tests, integration tests, and ensuring coverage type: bug Issue that causes incorrect or unexpected behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants