Skip to content

[QUERY] JMX metrics which aren't available initially, never recorded #1233

@bigredmachine

Description

@bigredmachine

Query/Question
I'm finding that some JMX metrics I'm trying to record via Java Application Insights aren't being recorded.
The one's which aren't being recorded are the one's which take the longest to appear in JConsole after the application starts up.
I've turned on the SDKLogger to see what's happening, and getting the following error
'java.lang.IllegalArgumentException: Cannot find object name 'solr:dom1=core,dom2=YOURCOLLECTIONNAME,dom3=shard1,dom4=replica_n1,category=SEARCHER,scope=searcher,name=numDocs'', The PC will be ignored

The application I'm trying to record JMX metric for is Solr.
These collection specific JMX metrics aren't loaded until the collection is loaded, which can take a minute or two depending on how many collections there are to load.

So what I think is happening is Application Insights is trying to load the JMX value before it exists, finding it doesn't exist, and never retries to get the JMX later after is has been created.

Is there a way to allow for the JMX metric to take some time to appear, before giving up that it doesn't exist?

I'd also be interested in working out how to programmatically adding to the JMX metrics being recorded, via some extension/plugin, for this application I don't control the source for.
E.g. add them when they become available, or add dynamic JMX metrics which names change each time the application runs, e.g.
solr:dom1=jetty,name0=org,name1=eclipse,name2=jetty,name3=util,name4=thread,name5=QueuedThreadPool,name6=qtpRANDOMNUMBER,name=jobs
And add them for all collections, shards, replicas, without having to maintain a configuration file.
Or extracting the values from the Solr Metrics API into application insights, as another alternative.
https://lucene.apache.org/solr/guide/8_5/metrics-reporting.html
Possibly I could get them into Application Insights via the SLF4J Reporter, and LF4J java application insights appender. I'm thinking though this might just come through as traces, but would like to come through as performance counters.

Why is this not a Bug or a feature Request?
I'm not sure if this is a bug, or if this is supported?
Happy for this to be changed depending on what it is.

Setup (please complete the following information if applicable):

OS: Windows 10
IDE : N/A
Version of the Library used - Application Insights Java - 2.6.0
applicationinsights-core-2.6.0.jar
applicationinsights-logging-log4j2-2.6.0.jar
Solr - 8.1.1
JConsole

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions