common/TrackedOp: rename and raise prio of slow op perfcounter by YiteGu · Pull Request #52068 · ceph/ceph

YiteGu · 2023-06-15T04:33:26Z

TrackedOp is a common class, mon and mds alse use it.
Set prio level of l_trackedop_slow_op_count to PRIO_USEFUL, we can get change of slow op of daemon from prometheus metrics, it's very interesting

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows

YiteGu · 2023-06-15T04:36:58Z

# ceph -c ceph.conf daemon asok/osd.0.asok perf dump | grep -A 5 trackedop
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
    "trackedop": {
        "slow_ops_count": 0
    }
}
# ceph -c ceph.conf daemon asok/mon.a.asok perf dump | grep -A 5 trackedop
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
    "trackedop": {
        "slow_ops_count": 0
    }
}
# ./ceph -c ceph.conf daemon asok/mds.c.asok perf dump | grep -A 5 trackedop
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
    "trackedop": {
        "slow_ops_count": 0
    }
}

YiteGu · 2023-06-15T04:38:02Z

from metrics:

# HELP ceph_trackedop_slow_ops_count Number of operations taking over ten second
# TYPE ceph_trackedop_slow_ops_count counter
ceph_trackedop_slow_ops_count{ceph_daemon="mds.c"} 0.0
ceph_trackedop_slow_ops_count{ceph_daemon="mon.a"} 0.0
ceph_trackedop_slow_ops_count{ceph_daemon="mon.b"} 0.0
ceph_trackedop_slow_ops_count{ceph_daemon="mon.c"} 0.0
ceph_trackedop_slow_ops_count{ceph_daemon="osd.0"} 0.0
ceph_trackedop_slow_ops_count{ceph_daemon="osd.1"} 0.0
ceph_trackedop_slow_ops_count{ceph_daemon="osd.2"} 0.0

YiteGu · 2023-06-15T04:41:26Z

@rzarzynski @athanatos have a code review, urgent

1. TrackedOp is a common class, mon and mds alse use it. 2. Set prio level of l_trackedop_slow_op_count to PRIO_USEFUL, we can get change of slow op of daemon from prometheus metrics, it's very interesting Signed-off-by: Yite Gu <yitegu0@gmail.com>

athanatos · 2023-06-20T18:27:50Z

So, the main change here is to set the priority for l_osd_slow_op_count to PRIO_USEFUL. Seems reasonable to me. @rzarzynski @batrick Does this seem reasonable to you?

batrick

Please rewrite commit title as: common/TrackedOp: rename and raise prio of slow op perfcounter

batrick · 2023-06-20T19:44:29Z

So, the main change here is to set the priority for l_osd_slow_op_count to PRIO_USEFUL. Seems reasonable to me. @rzarzynski @batrick Does this seem reasonable to you?

Yup!

rzarzynski

LGTM assuming the Patrick's review gets addressed.

src/common/TrackedOp.h

YiteGu · 2023-06-21T01:23:50Z

@batrick Rewrite commit title done

YiteGu · 2023-08-01T07:11:49Z

results show

github-actions · 2023-10-30T09:01:41Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

YiteGu · 2023-10-31T02:19:07Z

Hi @ljflores this pr need testing

github-actions · 2023-12-30T04:01:27Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

github-actions · 2024-03-02T19:01:28Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

github-actions · 2024-04-01T20:01:31Z

This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution!

YiteGu · 2024-07-01T03:30:39Z

@rzarzynski I think we need to fix this problem, otherwise, the slow op of mon and mds will be represented by the keyword osd, this will cause ambiguity

batrick · 2024-07-22T16:41:12Z

ping @ljflores @rzarzynski @yuriw please QA.

rzarzynski · 2024-07-23T07:02:20Z

jenkins test make check

rzarzynski · 2024-07-23T07:03:12Z

needs-qa is already assigned.

@yuriw, @ljflores: let's take this into the very next QA batch. Pinging also in Slack.

rzarzynski · 2024-07-23T10:59:40Z

Looks unrelated at first glance:

Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/mgr/dashboard/frontend/node-env/bin/nodeenv", line 8, in 
    sys.exit(main())
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/mgr/dashboard/frontend/node-env/lib/python3.10/site-packages/nodeenv.py", line 1130, in main
    create_environment(env_dir, args)
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/mgr/dashboard/frontend/node-env/lib/python3.10/site-packages/nodeenv.py", line 1006, in create_environment
    install_node(env_dir, src_dir, args)
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/mgr/dashboard/frontend/node-env/lib/python3.10/site-packages/nodeenv.py", line 763, in install_node
    install_node_wrapped(env_dir, src_dir, args)
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/mgr/dashboard/frontend/node-env/lib/python3.10/site-packages/nodeenv.py", line 786, in install_node_wrapped
    download_node_src(node_url, src_dir, args)
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/mgr/dashboard/frontend/node-env/lib/python3.10/site-packages/nodeenv.py", line 618, in download_node_src
    dl_contents = _download_node_file(node_url)
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/mgr/dashboard/frontend/node-env/lib/python3.10/site-packages/nodeenv.py", line 610, in _download_node_file
    raise e
  File "/home/jenkins-build/build/workspace/ceph-pull-requests/build/src/pybind/mgr/dashboard/frontend/node-env/lib/python3.10/site-packages/nodeenv.py", line 602, in _download_node_file
    return io.BytesIO(urlopen(node_url).read())
  File "/usr/lib/python3.10/http/client.py", line 482, in read
    s = self._safe_read(self.length)
  File "/usr/lib/python3.10/http/client.py", line 633, in _safe_read
    raise IncompleteRead(data, amt-len(data))
http.client.IncompleteRead: IncompleteRead(12623872 bytes read, 34097368 more expected)

rzarzynski · 2024-07-23T10:59:50Z

jenkins test make check

ljflores · 2024-07-31T05:31:12Z

Rados approved: https://tracker.ceph.com/projects/rados/wiki/MAIN#httpstrackercephcomissues67090

github-actions bot added the common label Jun 15, 2023

athanatos requested review from athanatos, batrick and rzarzynski June 20, 2023 18:27

athanatos added the needs-qa label Jun 20, 2023

athanatos approved these changes Jun 20, 2023

View reviewed changes

batrick requested changes Jun 20, 2023

View reviewed changes

rzarzynski approved these changes Jun 20, 2023

View reviewed changes

src/common/TrackedOp.h Show resolved Hide resolved

YiteGu force-pushed the fix-trackedop-perfcounter branch from 54bcdc1 to e1a0284 Compare June 21, 2023 01:20

YiteGu changed the title ~~osd/TrackedOp: Redefine TrackedOp perfcounter~~ common/TrackedOp: rename and raise prio of slow op perfcounter Jun 21, 2023

YiteGu requested a review from batrick June 21, 2023 01:24

batrick approved these changes Jun 21, 2023

View reviewed changes

YiteGu mentioned this pull request Jun 21, 2023

mds/MDSRank: Add set_history_slow_op_size_and_threshold for op_tracker #52081

Merged

14 tasks

github-actions bot added the stale label Oct 30, 2023

github-actions bot removed the stale label Oct 31, 2023

github-actions bot added the stale label Dec 30, 2023

athanatos removed the stale label Jan 2, 2024

github-actions bot added the stale label Mar 2, 2024

github-actions bot closed this Apr 1, 2024

batrick reopened this Jul 22, 2024

batrick added core and removed stale labels Jul 22, 2024

yuriw added the wip-yuri5-testing label Jul 23, 2024

yuriw merged commit 815fd8c into ceph:main Jul 31, 2024

This was referenced Aug 17, 2024

reef: common/TrackedOp: rename and raise prio of slow op perfcounter #59280

Merged

squid: common/TrackedOp: rename and raise prio of slow op perfcounter #59281

Closed

Conversation

YiteGu commented Jun 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution Guidelines

Checklist

Uh oh!

YiteGu commented Jun 15, 2023

Uh oh!

YiteGu commented Jun 15, 2023

Uh oh!

YiteGu commented Jun 15, 2023

Uh oh!

athanatos commented Jun 20, 2023

Uh oh!

batrick left a comment

Choose a reason for hiding this comment

Uh oh!

batrick commented Jun 20, 2023

Uh oh!

rzarzynski left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

YiteGu commented Jun 21, 2023

Uh oh!

YiteGu commented Aug 1, 2023

Uh oh!

github-actions bot commented Oct 30, 2023

Uh oh!

YiteGu commented Oct 31, 2023

Uh oh!

github-actions bot commented Dec 30, 2023

Uh oh!

github-actions bot commented Mar 2, 2024

Uh oh!

github-actions bot commented Apr 1, 2024

Uh oh!

YiteGu commented Jul 1, 2024

Uh oh!

batrick commented Jul 22, 2024

Uh oh!

rzarzynski commented Jul 23, 2024

Uh oh!

rzarzynski commented Jul 23, 2024

Uh oh!

rzarzynski commented Jul 23, 2024

Uh oh!

rzarzynski commented Jul 23, 2024

Uh oh!

ljflores commented Jul 31, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

YiteGu commented Jun 15, 2023 •

edited

Loading