Skip to content

common/TrackedOp: do not count the ops marked as nowarn#58125

Merged
batrick merged 1 commit intoceph:mainfrom
lxbsz:wip-fix-optracker
Jun 23, 2024
Merged

common/TrackedOp: do not count the ops marked as nowarn#58125
batrick merged 1 commit intoceph:mainfrom
lxbsz:wip-fix-optracker

Conversation

@lxbsz
Copy link
Member

@lxbsz lxbsz commented Jun 19, 2024

If an op is marked as nowarn then it won't be counted as the slow requests, but currently it will count the initiated time when iterating the inflight ops.

For example:

[WRN] : 1 slow requests, 1 included below; oldest blocked for > 38.764892 secs
[WRN] : slow request 33.875059 seconds old, received at 2024-06-17T14:14:34.228261+0000: client_request(client.78109915:11369251 mkdir #0x1008ecedea2/chk-89588 2024-06-17T14:14:34.097825+0000 caller_uid=1002960000, caller_gid=0{0,1002960000,}) currently failed to wrlock, waiting

The oldest blocked request is 38.764892 old, but the oldest slow request reported is 33.875059 old.

This will fix the old commit from #21715.

Fixes: https://tracker.ceph.com/issues/66557

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

@lxbsz lxbsz requested review from a team, batrick, gregsfortytwo and vshankar June 19, 2024 04:06
@lxbsz lxbsz force-pushed the wip-fix-optracker branch 2 times, most recently from 91c3a48 to a8c1bf9 Compare June 19, 2024 06:03
@leonid-s-usov
Copy link
Contributor

@batrick we had to introduce a new field continuous in the quiesce epic to achieve the same, right? Should we review?

@lxbsz lxbsz force-pushed the wip-fix-optracker branch from a8c1bf9 to 86ab7a3 Compare June 19, 2024 08:34
Copy link
Member

@batrick batrick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

otherwise lgtm

utime_t oldest_op_tmp =
sdata->ops_in_flight_sharded.front().get_initiated();
for (auto& op : sdata->ops_in_flight_sharded) {
if (!op.warn_interval_multiplier)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!op.warn_interval_multiplier)
if (!op.warn_interval_multiplier || op->is_continuous())

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Will fix it.

@batrick
Copy link
Member

batrick commented Jun 19, 2024

@batrick we had to introduce a new field continuous in the quiesce epic to achieve the same, right? Should we review?

We could consolidate them, yes.

If an op is marked as nowarn then it won't be counted as the slow
requests, but currently it will count the initiated time when
iterating the inflight ops.

For example:

[WRN] : 1 slow requests, 1 included below; oldest blocked for > 38.764892 secs
[WRN] : slow request 33.875059 seconds old, received at 2024-06-17T14:14:34.228261+0000: client_request(client.78109915:11369251 mkdir #0x1008ecedea2/chk-89588 2024-06-17T14:14:34.097825+0000 caller_uid=1002960000, caller_gid=0{0,1002960000,}) currently failed to wrlock, waiting

The oldest blocked request is 38.764892 old, but the oldest slow
request reported is 33.875059 old.

Fixes: commit e4160d7 ("mds: don't report slow request for blocked filelock request")
Fixes: https://tracker.ceph.com/issues/66557
Signed-off-by: Xiubo Li <xiubli@redhat.com>
@lxbsz lxbsz force-pushed the wip-fix-optracker branch from 86ab7a3 to 8debaf5 Compare June 20, 2024 00:25
@batrick
Copy link
Member

batrick commented Jun 21, 2024

jenkins test api

@batrick
Copy link
Member

batrick commented Jun 21, 2024

jenkins test make check arm64

@batrick
Copy link
Member

batrick commented Jun 21, 2024

jenkins test make check arm64

2 similar comments
@batrick
Copy link
Member

batrick commented Jun 21, 2024

jenkins test make check arm64

@batrick
Copy link
Member

batrick commented Jun 22, 2024

jenkins test make check arm64

@batrick
Copy link
Member

batrick commented Jun 22, 2024

This PR is under test in https://tracker.ceph.com/issues/66609.

@batrick
Copy link
Member

batrick commented Jun 22, 2024

jenkins test make check arm64

@batrick
Copy link
Member

batrick commented Jun 23, 2024

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants