Update counter cache only if the relation is actually saved.#23357
Update counter cache only if the relation is actually saved.#23357piotrj wants to merge 1 commit intorails:masterfrom
Conversation
|
Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @arthurnn (or someone else) soon. If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes. Please see the contribution instructions for more information. |
|
@arthurnn any thoughts on that. Should someone else look at this? There are quite a few issues related to that reported so I guess it would be nice to get this shipped. |
|
Would someone give an estimate when this will be reviewed? I am experiencing this issue as well. |
|
Do we still need this, after #24713 was merged in? |
|
@maclover7 yes we still need this. Moreover, probably after this is merged the other fix would be irrelevant since we don't call update_counters_on_replace here. Also, this PR fixes counter caches for polymorphic associations. Can we get someone to look into this and merge it? |
|
I think you would need to rebase first, and fix the current merge conflicts ;) |
|
@maclover7 done. |
|
@maclover7 @arthurnn hey guys, I just rebased that again so it's clean to merge. Any chance we can close this? :) |
|
@felixbuenemann quite a lot of stuff changed since I introduced this fix. I have to investigate whether my fix still makes sense. Will try to do rebase by the end of the week. If the fix is still valid I will add your issue. Thanks for the comment. |
|
@felixbuenemann rebased and added your commit. You were right, this fix is still needed. |
Fixes rails#18988, rails#22602, rails#23265, rails#28203 *** Description of the problem So far the counter cached was incremented when one of two things happened in following scenario. ``` class Post < ActiveRecord::Base has_many :coments end class Comment < ActiveRecord::base belongs_to :post, counter_cache: true end ``` 1. comment.post = post was called. This would increment the counter cache for post in the db 2. Comment was saved with updated post_id and the counter cache is incremented in update callback This solution (especially the fact that we did counter cache increment in situation rails#1) had few problems: 1. We bumped the counter cache even if in the end the Comment was not saved. For example if something like this happened: ``` comment.post = post raise SomeException ``` then the comment wouldn't have post_id, but post would end up with 2. If someone did the following ``` comment.post = post comment.save! ``` Post would end up with comments_count = 2 because the counter cache would get bumped on both of conditions above. 3. The bump in situation rails#1 was made outside of transaction that saves the comment which means we would make unnecessary call to db. 4. comment.post = post and comment.post_id = post.id are not consistent. Former bumps the bumper cache while the latter does not. *** Solution This commit removes the counter cache increment on association assignment. This means that comment.post = post won't bump the counter. It will only happen when the comment will get saved. This will ensure more consistency of data in DB. Other changes that are introduced in this commit: 1. Counter cache incrementer/decrementer can work with foreign_keys other than the primary_key of the model. That's why methods updates_counters_for_key, increment_counter_for_key and decrement_counter_for_key has been introduced. 2. The update callback that bumps counter cache will properly increment counter cache for polymorphic association even if the only thing that was changed was type of the related object but it happened to have the same id as the previously related object. Notes about chaned tests: 1. In test_belongs_to_counter_with_assigning_nil I changed the models under test as Comment's post_id is non nullable therefore comment could never have been saved with post_id = nil. Moreover this test in its original form shown how flawed the previous behavior of counter cache was. When nil was assigned to comment.post, that post's comments_count would get decremented, but in the end it would be possible to save that comment and it would result in inconsistent data. 2. test_counter_cache_with_custom_column_name - the relation of SillyReply is basically broken. It uses the same foreign_key "parent_id" as the Reply (its superclass) topic relation. That's why I used DogLovers and Dogs for the test of custom names for counter caches. 3. Other tests have been updated to actually save the objects in order to bump counter cache.
This is a 4th attempt to make counter cache transactional completely. Past attempts: rails#9236, rails#14849, rails#23357. All existing counter cache issues (increment/decrement twice, lost increment) are caused due to updating counter cache on the outside of the record saving transaction by assigning belongs_to record, even though assigning that doesn't cause the record saving. We have the `@_after_replace_counter_called` guard condition to mitigate double increment/decrement issues, but we can't completely prevent that inconsistency as long as updating counter cache on the outside of the transaction, since saving the record is not always happened after that. We already have handling counter cache after create/update/destroy, https://github.com/rails/rails/blob/1b90f614b1b3d06b7f02a8b9ea6cd84f15d58643/activerecord/lib/active_record/counter_cache.rb#L162-L189 https://github.com/rails/rails/blob/1b90f614b1b3d06b7f02a8b9ea6cd84f15d58643/activerecord/lib/active_record/associations/builder/belongs_to.rb#L33-L59 so just removing assigning logic on the belongs_to association makes counter cache transactional completely. Closes rails#14849. Closes rails#23357. Closes rails#31493. Closes rails#31494. Closes rails#32372. Closes rails#33113. Closes rails#33117 Closes rails#33129. Closes rails#33458.
Fixes #18988, #22602, #23265, #28203
Description of the problem
So far the counter cached was incremented when one of two things
happened in following scenario.
comment.post = postwas called. This would increment the countercache for post in the db
incremented in update callback
This solution (especially the fact that we did counter cache increment in situation #1) had few problems:
saved. For example if something like this happened:
then the comment wouldn't have post_id, but post would end up with
Post would end up with comments_count = 2 because the counter cache
would get bumped on both of conditions above.
the comment which means we would make unnecessary call to db.
Former bumps the bumper cache while the latter does not.
Solution
This commit removes the counter cache increment on association
assignment. This means that comment.post = post won't bump the counter.
It will only happen when the comment will get saved. This will ensure
more consistency of data in DB.
Other changes that are introduced in this commit:
other than the primary_key of the model. That's why methods
updates_counters_for_key, increment_counter_for_key and
decrement_counter_for_key has been introduced.
counter cache for polymorphic association even if the only thing that
was changed was type of the related object but it happened to have the
same id as the previously related object.
Notes about chaned tests:
under test as Comment's post_id is non nullable therefore comment could
never have been saved with post_id = nil. Moreover this test in its
original form shown how flawed the previous behavior of counter cache
was. When nil was assigned to comment.post, that post's comments_count
would get decremented, but in the end it would be possible to save that
comment and it would result in inconsistent data.
SillyReply is basically broken. It uses the same foreign_key "parent_id"
as the Reply (its superclass) topic relation. That's why I used
DogLovers and Dogs for the test of custom names for counter caches.
to bump counter cache.