Skip to content

add prometheus-emitter to distribution#11812

Merged
clintropolis merged 2 commits intoapache:masterfrom
dungdm93:docker-prometheus
Oct 26, 2021
Merged

add prometheus-emitter to distribution#11812
clintropolis merged 2 commits intoapache:masterfrom
dungdm93:docker-prometheus

Conversation

@dungdm93
Copy link
Contributor

@dungdm93 dungdm93 commented Oct 19, 2021

prometheus is de-facto monitoring stack in cloud native/container environment like Kubernetes.
It's very helpful if druid docker image ship with prometheus-emitter extension by default.

Signed-off-by: Đặng Minh Dũng dungdm93@live.com

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>
Copy link
Member

@clintropolis clintropolis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the fix 👍

This doesn't need to be solved in this PR, but I wonder if there is some check we could add to CI that makes sure extensions are included in either the standard build profile or the bundle contrib extensions profile.

I guess 'druid-testing-tools' is defined primarily for use in testing and so i'm not sure would ever make sense to include in a typical bundle, so such a check would either need a way to handle ignored extensions, or we would include a 3rd profile for non-typically distributed extensions perhaps.

<argument>-c</argument>
<argument>org.apache.druid.extensions.contrib:statsd-emitter</argument>
<argument>-c</argument>
<argument>org.apache.druid.extensions.contrib:prometheus-emitter</argument>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch.

I did a quick check and it looks like we are also missing druid-momentsketch, would you mind adding it too?

druid-testing-tools is also not in any profile, but I'm not sure it actually makes sense to include anyway so we should ignore it for now, because its primary purpose is to aid in writing integration tests since it includes mechanisms to control query execution time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @clintropolis, druid-momentsketch is added.

@clintropolis
Copy link
Member

closing and re-opening to try to get this picked up by travis

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>
@dungdm93
Copy link
Contributor Author

Hello @clintropolis
Please help me rerun travis CI

@dungdm93
Copy link
Contributor Author

@clintropolis all checks are green

@clintropolis clintropolis merged commit 4baebb2 into apache:master Oct 26, 2021
@dungdm93 dungdm93 deleted the docker-prometheus branch October 26, 2021 07:52
jon-wei pushed a commit to jon-wei/druid that referenced this pull request Nov 22, 2021
* fix type

* Revert "Fix Keyclock auth integration test based on upstream changes (apache#622)"

This reverts commit c1083fb.

* Bump netty4 to 4.1.68; suppress CVE-2021-37136 and CVE-2021-37137 for netty3 (apache#11844)

* bump netty4 to 4.1.68

* suppress CVE-2021-37136 and CVE-2021-37137 for netty3

* license

* add `prometheus-emitter` to distribution (apache#11812)

* add `prometheus-emitter` to distribution

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>

* add `druid-momentsketch` to distribution

Signed-off-by: Đặng Minh Dũng <dungdm93@live.com>

* Web console: update typescript 4.4 for faster build speeds (apache#11725)

* update typescript

* do not show pagination when there is only one page

* update snapshots

* fix pagination

* Remove OffheapIncrementalIndex and clarify aggregator thread-safety needs. (apache#11124)

* Remove OffheapIncrementalIndex and clarify aggregator thread-safety needs.

This patch does the following:

- Removes OffheapIncrementalIndex.
- Clarifies that Aggregators are required to be thread safe.
- Clarifies that BufferAggregators and VectorAggregators are not
  required to be thread safe.
- Removes thread safety code from some DataSketches aggregators that
  had it. (Not all of them did, and that's OK, because it wasn't necessary
  anyway.)
- Makes enabling "useOffheap" with groupBy v1 an error.

Rationale for removing the offheap incremental index:

- It is only used in one rare scenario: groupBy v1 (which is non-default)
  in "useOffheap" mode (also non-default). So you have to go pretty deep
  into the wilderness to get this code to activate in production. It is
  never used during ingestion.
- Its existence complicates developer efforts to reason about how
  aggregators get used, because the way it uses buffer aggregators is so
  different from how every other query engine uses them.
- It doesn't have meaningful testing.

By the way, I do believe that the given way the offheap incremental index
works, it actually didn't require buffer aggregators to be thread-safe.
It synchronizes on "aggregate" and doesn't call "get" until it has
stopped calling "aggregate". Nevertheless, this is a bother to think about,
and for the above reasons I think it makes sense to remove the code anyway.

* Remove things that are now unused.

* Revert removal of getFloat, getLong, getDouble from BufferAggregator.

* OAK-related warnings, suppressions.

* Unused item suppressions.

Co-authored-by: Đặng Minh Dũng <dungdm93@live.com>
Co-authored-by: Vadim Ogievetsky <vadim@ogievetsky.com>
Co-authored-by: Gian Merlino <gianmerlino@gmail.com>
@abhishekagarwal87 abhishekagarwal87 added this to the 0.23.0 milestone May 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants