Skip to content

in_tail: position_file: Define path based equalities on TargetInfo#3391

Closed
cosmo0920 wants to merge 3 commits into
masterfrom
define-target-info-equalities
Closed

in_tail: position_file: Define path based equalities on TargetInfo#3391
cosmo0920 wants to merge 3 commits into
masterfrom
define-target-info-equalities

Conversation

@cosmo0920

@cosmo0920 cosmo0920 commented May 25, 2021

Copy link
Copy Markdown
Contributor

Signed-off-by: Hiroshi Hatake hatake@calyptia.com

Which issue(s) this PR fixes:
Related to #3365

What this PR does / why we need it:

In the previous Fluentd v1.11.x implementation uses path based tailing
keys.
We wouldn't rewrite tailing mechanism entirely.
And also we're implicitly assuming path based equality for TargetInfo in tailing logic.
We should handle TargetInfo with path based equality instead of path and
inode based equality.

Docs Changes:

No needed.

Release Note:

Same as title.

@cosmo0920 cosmo0920 requested review from ashie and kenhys May 25, 2021 04:44
@cosmo0920 cosmo0920 self-assigned this May 25, 2021
@cosmo0920 cosmo0920 force-pushed the define-target-info-equalities branch from 883eaa7 to 2c9b31f Compare May 25, 2021 04:49
@ashie

ashie commented May 25, 2021

Copy link
Copy Markdown
Member

Typo?

-In the previous Fluentd v1.11.x implementation uses patch based tailing keys.
+In the previous Fluentd v1.11.x implementation uses path based tailing keys.

In the previous Fluentd v1.11.x implementation uses path based tailing
keys.
We wouldn't rewrite tailing mechanism entirely.
And also we're implicitly assuming path based equality for TargetInfo in tailing logic.
We should handle TargetInfo with path based equality instead of path and
inode based equality.

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
@cosmo0920 cosmo0920 force-pushed the define-target-info-equalities branch from 2c9b31f to e516268 Compare May 25, 2021 04:51
@cosmo0920

Copy link
Copy Markdown
Contributor Author

Typo?

-In the previous Fluentd v1.11.x implementation uses patch based tailing keys.
+In the previous Fluentd v1.11.x implementation uses path based tailing keys.

Yep. This is typo. I'd fixed it.

@ashie

ashie commented May 25, 2021

Copy link
Copy Markdown
Member

Hmm, we should release v1.12.4 with this patch before releasing v1.13?

@ashie

ashie commented May 25, 2021

Copy link
Copy Markdown
Member

I'll create v1.12 branch.

@cosmo0920

Copy link
Copy Markdown
Contributor Author

Hmm, we should release v1.12.4 with this patch before releasing v1.13?

Yes, please.

@ashie

ashie commented May 25, 2021

Copy link
Copy Markdown
Member

I'll release v1.12.4 in a few days.

end

def hash
self.path.hash

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure follow_inodes feature but does it work well also on @follow_inodes?
I think that keys of @tails should be inode on @follow_inodes.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or paths are updated by update_watcher on rotation so paths are never duplicate?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll take a look on this.

@ashie ashie May 26, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still checking it too.
Probably above my concern is wrong, it should be path maybe.
Otherwise update_watcher can't discard old TailWather from @tails.

@ashie ashie May 26, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Memo:

Instead of overriding them, I thought using target_info.path as a key is more straightforward and understandable: @tails[target_info.path]

But it's also wrong.
TargetInfo is referred by each_keys so keys have to be TargetInfo.

@ashie ashie May 26, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, probably it's O.K. as a first aid but I think more refactoring is needed to make easy to understand in the future 🤔

@cosmo0920 cosmo0920 May 26, 2021

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of overriding them, I thought using target_info.path as a key is more straightforward and understandable: @tails[target_info.path]

But it's also wrong.
TargetInfo is referred by each_keys so keys have to be TargetInfo.

Yes! I'm also considering to use it to be made easier to understand, but overriding equality on TargetInfo is one of the simplest way to dispose discarded old TailWatcher correctly. (This strategy doesn't request more refactoring.)

t1.eql? t2
end
end

@kenhys kenhys May 26, 2021

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To emphasize the difference of eql? and equal? behavior, how about adding one more test case using assert_not_same t1, t2?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be done in ab36738 and 2f2214b.

cosmo0920 added 2 commits May 26, 2021 12:12
Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
…sub_test_case

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
@ashie ashie changed the base branch from master to v1.12 May 26, 2021 03:37
@ashie ashie changed the base branch from v1.12 to master May 26, 2021 03:37
@ashie

ashie commented May 26, 2021

Copy link
Copy Markdown
Member

I've created a pull request for v1.12 and merged already: #3393
I'll merge v1.12 branch to master after releasing v1.12.4, then close this PR.
Thank you for investigating it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants