Skip to content

mgr: templatize metrics collection interface#29214

Merged
liewegas merged 1 commit intoceph:masterfrom
vshankar:wip-templatize-metrics-collection
Dec 9, 2019
Merged

mgr: templatize metrics collection interface#29214
liewegas merged 1 commit intoceph:masterfrom
vshankar:wip-templatize-metrics-collection

Conversation

@vshankar
Copy link
Contributor

[This change was initially a part of fs top work where MDS forwards metrics information to the manager]

Metric collection for a ceph entity can inherit from MetricCollector class (see OSDPerfMetricCollector) which provides basic {add,remove}_query (templated) interface. process_reports is a virtual function which needs to be overridden to process MetricPayload variant (which is a part of MMgrReport::metric_report_message.

@vshankar vshankar added core cephfs Ceph File System labels Jul 23, 2019
@batrick batrick added mgr and removed core labels Jul 26, 2019
@vshankar
Copy link
Contributor Author

@vshankar switch to PR when this is ready for review

yep -- in a days time

@vshankar vshankar force-pushed the wip-templatize-metrics-collection branch from 50aed6b to 0bcf020 Compare July 31, 2019 12:48
@vshankar vshankar marked this pull request as ready for review July 31, 2019 12:57
@vshankar vshankar requested review from batrick and liewegas July 31, 2019 13:27
@batrick batrick requested a review from tchaikov July 31, 2019 19:01
@vshankar vshankar force-pushed the wip-templatize-metrics-collection branch from 0bcf020 to d5a9b0d Compare August 1, 2019 02:56
@tchaikov
Copy link
Contributor

tchaikov commented Aug 7, 2019

will do the review today! sorry for the latency.

@vshankar
Copy link
Contributor Author

vshankar commented Aug 7, 2019

will do the review today! sorry for the latency.

I'll push an update (+rebase) soon -- requesting hold on to reviews.

@vshankar vshankar force-pushed the wip-templatize-metrics-collection branch from d5a9b0d to cda07bd Compare August 7, 2019 09:21
@vshankar
Copy link
Contributor Author

vshankar commented Aug 7, 2019

@tchaikov rebased and updated. please take a look.

@vshankar
Copy link
Contributor Author

@tchaikov pinging for reviews

@tchaikov
Copy link
Contributor

@vshankar i am still reviewing it. will finish the review tomorrow.

Copy link
Contributor

@tchaikov tchaikov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am still reviewing this PR..

uint32_t stats_threshold = 0;

std::map<OSDPerfMetricQuery, OSDPerfMetricLimits> osd_perf_metric_queries;
boost::optional<std::map<OSDPerfMetricQuery, OSDPerfMetricLimits>> osd_perf_metric_queries;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not backward compatible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true -- I had made changes to MgrClient::_send_configure() to use boost::optional which I reverted since it wasn't backward compatible. Forgot to revert this bit.


uint64_t query_id;
bool notify = false;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto.

src/osd/OSD.cc Outdated
},
[this](std::map<OSDPerfMetricQuery, OSDPerfMetricReport> *reports) {
get_perf_reports(reports);
[this]() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit, could remove () if no parameters will be passed to this lambda.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice to know :)

@vshankar
Copy link
Contributor Author

i am still reviewing this PR..

@tchaikov checking back on this.

@vshankar vshankar force-pushed the wip-templatize-metrics-collection branch from cda07bd to 8d8f260 Compare October 16, 2019 11:46
@vshankar
Copy link
Contributor Author

@tchaikov sorry for the late update -- been tied up with other work.

rebased + comments addressed. please take a look.

@vshankar vshankar force-pushed the wip-templatize-metrics-collection branch from 8d8f260 to 1119c38 Compare November 18, 2019 10:28
@vshankar
Copy link
Contributor Author

@tchaikov @batrick -- rebased and updated.

@tchaikov tchaikov self-requested a review November 18, 2019 14:57
@batrick
Copy link
Member

batrick commented Dec 5, 2019

@vshankar please rebase and work with @tchaikov to QA this with the relevant RADOS suites. We really need to get this done with as it blocks your other ceph fs top PRs.

@vshankar
Copy link
Contributor Author

vshankar commented Dec 6, 2019

@vshankar please rebase and work with @tchaikov to QA this with the relevant RADOS suites. We really need to get this done with as it blocks your other ceph fs top PRs.

I'll push an update today -- @tchaikov requesting your review on this.

templatize metrics collection so as to reuse quering routines.
`MetricCollector` can be subclassed and along with implementing
` process_reports()` to process incoming metrics data.

also, generalize metrics data in `MMgrReport` and metric query
configuration in `MMgrConfigure`.

Signed-off-by: Venky Shankar <vshankar@redhat.com>
@vshankar vshankar force-pushed the wip-templatize-metrics-collection branch from 1119c38 to efcebe1 Compare December 6, 2019 05:13
@vshankar
Copy link
Contributor Author

vshankar commented Dec 6, 2019

rebased and updated

Copy link
Member

@batrick batrick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. Don't know if I fully understand some of the boost parts but the idea looks right.

liewegas added a commit that referenced this pull request Dec 9, 2019
* refs/pull/29214/head:
	mgr: templatize/generalize metrics collection interface

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
@liewegas liewegas merged commit efcebe1 into ceph:master Dec 9, 2019
@batrick
Copy link
Member

batrick commented Dec 9, 2019

\o/ thanks @liewegas

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants