Conversation
anuraaga
left a comment
There was a problem hiding this comment.
Thanks for this idea! I like the idea of exporting beans from a server, but I have a couple of high level questions before digging in.
| def loadMatches = otel.queryJmx("org.apache.cassandra.metrics:type=Storage,name=Load") | ||
| def load = loadMatches.first() | ||
|
|
||
| def lvr = otel.longValueRecorder( |
There was a problem hiding this comment.
I think a lot of the industry has switched from inline scripts to standard static configs, often generating them in something like terraform, mainly because of security dangers of inline scripts. Do you have any thoughts on the scripting vs config approach?
Putting on my infrastructure engineer hat, exporting metrics for arbitrary management beans seems like an awesome feature I'd want to enable, but I would be quite wary in a production environment due to the security concern (I have no problem generating configs with another tool though)
There was a problem hiding this comment.
This is a good point, and I make no pretense to speak authoritatively but only from personal preference. Config based jmx solutions like jmxtrans and collectd's GenericJMX are powerful but have few or no capabilities for aggregations or transformations, and are generally unexpressive. They are also isolated and without ways of producing custom metrics informed by mbean state. Manual instrumentation seems preferable in light of this, though improving the helper utility to support a more declarative style of instrumentation could be useful.
As far as security concerns they seem to be the same as a lot of dynamic configuration management utilities. Guidelines for restricting permissions via an access file and configuring the security manager could be helpful to reduce the potential for destructive purposes.
|
|
||
| As configured in the example, this metric extension will configure an otlp gRPC metric exporter | ||
| at the `otel.jmx.metrics.exporter.endpoint` and establish an MBean server connection using the | ||
| provided `otel.jmx.metrics.service.url`. After loading the Groovy script whose path is specified |
There was a problem hiding this comment.
I had an impression we would be exporting management beans of the current server in this module, not connecting to arbitrary JMX endpoints. The latter does seem nicely flexible, but that seems like a scraper - shouldn't it live in the opentelemetry collector?
There was a problem hiding this comment.
The api could be extended to support the local platform server for new jmx-based instrumentations in this project and run within the agent apart from the central groovy script utility.
You're right that this is mainly a scraper as is, and the goal is to have receivers in collector-contrib that are consumers of this utility. Since that's not a java project or instrumentation library I'm not sure if the core functionality belongs there.
This seems more akin to an instrumentation like the opentelemetry-java runtime metrics extension, which I've been told is migrating to this project. @bogdandrutu, @tigrannajaryan thoughts?
There was a problem hiding this comment.
The main issue is that the Collector is written in Go and it is basically impossible (or highly impractical) to interact with JMX from any language other than Java. Technically a standalone Java app could be put in the Collector repo, but either way there has to be something separate from the collector itself to get anything JMX. I think also the idea is that the collector could possibly manage this Java process right @rmfitzpatrick ?
There was a problem hiding this comment.
Given that it is technically not feasible to make this a native receiver/scraper inside the Collector I think the decision on where this code should live depends on whether this new tool can be useful independently from the Collector (looks like it can). If it does not require the Collector and can be used on its own then this repository is probably a better place.
Alternatively, we may decide to have a separate repository for this sort of collectors which do not fit the Collector repo or the Java repo.
80fb601 to
ae5a335
Compare
|
Closing this following SIG recommendation: open-telemetry/community#449 |
These changes introduce a new generic JMX Metrics groovy script runner and utilities capable of generating custom metrics for arbitrary Java applications with an accessible MBean server. As such, this is a standalone utility that doesn't incorporate the existing java agent. It was originally proposed in open-telemetry/opentelemetry-java#1523.
It currently only supports synchronous instruments but asynchronous support is intended to be added shortly.