Skip to content

Update counter cache only if the relation is actually saved.#23357

Closed
piotrj wants to merge 1 commit intorails:masterfrom
piotrj:issue_23265
Closed

Update counter cache only if the relation is actually saved.#23357
piotrj wants to merge 1 commit intorails:masterfrom
piotrj:issue_23265

Conversation

@piotrj
Copy link
Copy Markdown
Contributor

@piotrj piotrj commented Jan 30, 2016

Fixes #18988, #22602, #23265, #28203

Description of the problem
So far the counter cached was incremented when one of two things
happened in following scenario.

  class Post < ActiveRecord::Base
    has_many :coments
  end

  class Comment < ActiveRecord::base
    belongs_to :post, counter_cache: true
  end
  1. comment.post = post was called. This would increment the counter
    cache for post in the db
  2. Comment was saved with updated post_id and the counter cache is
    incremented in update callback

This solution (especially the fact that we did counter cache increment in situation #1) had few problems:

  • We bumped the counter cache even if in the end the Comment was not
    saved. For example if something like this happened:
    comment.post = post
    raise SomeException

then the comment wouldn't have post_id, but post would end up with

  • If someone did the following
    comment.post = post
    comment.save!

Post would end up with comments_count = 2 because the counter cache
would get bumped on both of conditions above.

  • The bump in situation 1 was made outside of transaction that saves
    the comment which means we would make unnecessary call to db.
  • comment.post = post and comment.post_id = post.id are not consistent.
    Former bumps the bumper cache while the latter does not.

Solution
This commit removes the counter cache increment on association
assignment. This means that comment.post = post won't bump the counter.
It will only happen when the comment will get saved. This will ensure
more consistency of data in DB.

Other changes that are introduced in this commit:

  1. Counter cache incrementer/decrementer can work with foreign_keys
    other than the primary_key of the model. That's why methods
    updates_counters_for_key, increment_counter_for_key and
    decrement_counter_for_key has been introduced.
  2. The update callback that bumps counter cache will properly increment
    counter cache for polymorphic association even if the only thing that
    was changed was type of the related object but it happened to have the
    same id as the previously related object.

Notes about chaned tests:

  1. In test_belongs_to_counter_with_assigning_nil I changed the models
    under test as Comment's post_id is non nullable therefore comment could
    never have been saved with post_id = nil. Moreover this test in its
    original form shown how flawed the previous behavior of counter cache
    was. When nil was assigned to comment.post, that post's comments_count
    would get decremented, but in the end it would be possible to save that
    comment and it would result in inconsistent data.
  2. test_counter_cache_with_custom_column_name - the relation of
    SillyReply is basically broken. It uses the same foreign_key "parent_id"
    as the Reply (its superclass) topic relation. That's why I used
    DogLovers and Dogs for the test of custom names for counter caches.
  3. Other tests have been updated to actually save the objects in order
    to bump counter cache.

@rails-bot
Copy link
Copy Markdown

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @arthurnn (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@piotrj
Copy link
Copy Markdown
Contributor Author

piotrj commented Mar 23, 2016

@arthurnn any thoughts on that. Should someone else look at this? There are quite a few issues related to that reported so I guess it would be nice to get this shipped.

@AlexVPopov
Copy link
Copy Markdown

Would someone give an estimate when this will be reviewed? I am experiencing this issue as well.

@maclover7
Copy link
Copy Markdown
Contributor

Do we still need this, after #24713 was merged in?

@piotrj
Copy link
Copy Markdown
Contributor Author

piotrj commented May 3, 2016

@maclover7 yes we still need this. Moreover, probably after this is merged the other fix would be irrelevant since we don't call update_counters_on_replace here.

Also, this PR fixes counter caches for polymorphic associations.

Can we get someone to look into this and merge it?

@maclover7
Copy link
Copy Markdown
Contributor

I think you would need to rebase first, and fix the current merge conflicts ;)

@piotrj
Copy link
Copy Markdown
Contributor Author

piotrj commented May 3, 2016

@maclover7 done.

@piotrj
Copy link
Copy Markdown
Contributor Author

piotrj commented Jun 23, 2016

@maclover7 @arthurnn hey guys, I just rebased that again so it's clean to merge. Any chance we can close this? :)

@felixbuenemann
Copy link
Copy Markdown
Contributor

@piotrj Could you add issue #28203 to the list of issues which this PR fixes?

@piotrj
Copy link
Copy Markdown
Contributor Author

piotrj commented Feb 28, 2017

@felixbuenemann quite a lot of stuff changed since I introduced this fix. I have to investigate whether my fix still makes sense. Will try to do rebase by the end of the week. If the fix is still valid I will add your issue. Thanks for the comment.

@piotrj
Copy link
Copy Markdown
Contributor Author

piotrj commented Mar 3, 2017

@felixbuenemann rebased and added your commit. You were right, this fix is still needed.
@arthurnn any chance to get this merged? It's been quite some time for this to mature

Fixes rails#18988, rails#22602, rails#23265, rails#28203

*** Description of the problem
So far the counter cached was incremented when one of two things
happened in following scenario.

```
  class Post < ActiveRecord::Base
    has_many :coments
  end

  class Comment < ActiveRecord::base
    belongs_to :post, counter_cache: true
  end
```

1. comment.post = post was called. This would increment the counter
   cache for post in the db
2. Comment was saved with updated post_id and the counter cache is
   incremented in update callback

This solution (especially the fact that we did counter cache increment
in situation rails#1) had few problems:

1. We bumped the counter cache even if in the end the Comment was not
   saved. For example if something like this happened:

```
    comment.post = post
    raise SomeException
```

  then the comment wouldn't have post_id, but post would end up with

2. If someone did the following
```
    comment.post = post
    comment.save!
```

   Post would end up with comments_count = 2 because the counter cache
   would get bumped on both of conditions above.

3. The bump in situation rails#1 was made outside of transaction that saves
the comment which means we would make unnecessary call to db.

4. comment.post = post and comment.post_id = post.id are not consistent.
Former bumps the bumper cache while the latter does not.

*** Solution
This commit removes the counter cache increment on association
assignment. This means that comment.post = post won't bump the counter.
It will only happen when the comment will get saved. This will ensure
more consistency of data in DB.

Other changes that are introduced in this commit:
1. Counter cache incrementer/decrementer can work with foreign_keys
other than the primary_key of the model. That's why methods
updates_counters_for_key, increment_counter_for_key and
decrement_counter_for_key has been introduced.

2. The update callback that bumps counter cache will properly increment
counter cache for polymorphic association even if the only thing that
was changed was type of the related object but it happened to have the
same id as the previously related object.

Notes about chaned tests:
1. In test_belongs_to_counter_with_assigning_nil I changed the models
under test as Comment's post_id is non nullable therefore comment could
never have been saved with post_id = nil. Moreover this test in its
original form shown how flawed the previous behavior of counter cache
was. When nil was assigned to comment.post, that post's comments_count
would get decremented, but in the end it would be possible to save that
comment and it would result in inconsistent data.

2. test_counter_cache_with_custom_column_name - the relation of
SillyReply is basically broken. It uses the same foreign_key "parent_id"
as the Reply (its superclass) topic relation. That's why I used
DogLovers and Dogs for the test of custom names for counter caches.

3. Other tests have been updated to actually save the objects in order
to bump counter cache.
@arthurnn arthurnn assigned matthewd and rafaelfranca and unassigned arthurnn Mar 18, 2017
kamipo added a commit to kamipo/rails that referenced this pull request Sep 18, 2018
This is a 4th attempt to make counter cache transactional completely.

Past attempts: rails#9236, rails#14849, rails#23357.

All existing counter cache issues (increment/decrement twice, lost
increment) are caused due to updating counter cache on the outside of
the record saving transaction by assigning belongs_to record, even
though assigning that doesn't cause the record saving.

We have the `@_after_replace_counter_called` guard condition to mitigate
double increment/decrement issues, but we can't completely prevent that
inconsistency as long as updating counter cache on the outside of the
transaction, since saving the record is not always happened after that.

We already have handling counter cache after create/update/destroy,

https://github.com/rails/rails/blob/1b90f614b1b3d06b7f02a8b9ea6cd84f15d58643/activerecord/lib/active_record/counter_cache.rb#L162-L189
https://github.com/rails/rails/blob/1b90f614b1b3d06b7f02a8b9ea6cd84f15d58643/activerecord/lib/active_record/associations/builder/belongs_to.rb#L33-L59

so just removing assigning logic on the belongs_to association makes
counter cache transactional completely.

Closes rails#14849.
Closes rails#23357.
Closes rails#31493.
Closes rails#31494.
Closes rails#32372.
Closes rails#33113.
Closes rails#33117
Closes rails#33129.
Closes rails#33458.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants