feat: Database_observability: expose exclude_schemas and exclude_databases settings#5334
Conversation
| | ------------------------------ | -------------- | ------------------------------------------------------------------------------- | ------- | -------- | | ||
| | `collect_interval` | `duration` | How frequently to collect information from database. | `"1m"` | no | | ||
| | `explain_plan_exclude_schemas` | `list(string)` | List of schemas to exclude from explain plan collection. | | no | | ||
| | `exclude_schemas` | `list(string)` | List of schemas to exclude from explain plan collection. | `[]` | no | |
There was a problem hiding this comment.
I think this is ok to rename since it's never been wired, although technically a breaking change.
There was a problem hiding this comment.
Why was this not wired up before?
There was a problem hiding this comment.
Nonetheless - I think this can be set as a top level argument given it may be applicable to all collectors.
| |---------------------|----------------|------------------------------------------------------|---------|----------| | ||
| | `collect_interval` | `duration` | How frequently to collect information from database. | `"1m"` | no | | ||
| | `per_collect_ratio` | `float64` | The ratio of queries to collect explain plans for. | `1.0` | no | | ||
| | `exclude_schemas` | `list(string)` | Schemas to exclude from explain plans. | `[]` | no | |
There was a problem hiding this comment.
Same comment about renaming as for mysql.
|
💻 Deploy preview available (fix: Database_observability: expose 'exclude_schemas' for explain plans): |
exclude_schemas for explain plans
5b2354c to
9fff639
Compare
|
💻 Deploy preview available (fix: Database_observability: expose |
Correctly expose and wire setting to exclude schemas from explain plans collection.
9fff639 to
8d51c60
Compare
exclude_schemas for explain plansexclude_schemas and exclude_databases settings
| | Name | Type | Description | Default | Required | | ||
| | ------------------------------ | -------------- | ------------------------------------------------------------------------------- | ------- | -------- | | ||
| | `collect_interval` | `duration` | How frequently to collect information from database. | `"1m"` | no | | ||
| | `explain_plan_exclude_schemas` | `list(string)` | List of schemas to exclude from explain plan collection. | | no | |
There was a problem hiding this comment.
Can be deleted as it was not wired
docs/sources/reference/components/database_observability/database_observability.mysql.md
Outdated
Show resolved
Hide resolved
|
💻 Deploy preview deleted (feat: Database_observability: expose |
This MR contains the following updates: | Package | Update | Change | |---|---|---| | [grafana/alloy](https://github.com/grafana/alloy) | minor | `v1.12.2` → `v1.13.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>grafana/alloy (grafana/alloy)</summary> ### [`v1.13.0`](https://github.com/grafana/alloy/blob/HEAD/CHANGELOG.md#1130-2026-02-05) [Compare Source](grafana/alloy@v1.12.2...v1.13.0) ##### ⚠ BREAKING CHANGES - **otelcol:** Upgrade to OTel Collector v0.142.0 - **otelcol.receiver.kafka:** The global `topic` attribute has been deleted; use the `topics` attributes inside the `logs`, `metrics`, and `traces` blocks instead. - `otelcol.exporter` > `sending_queue` > `batch` > `min_size` changed from `8192` to `2000` and `max_size` changed from `0` to `3000` ##### Features 🌟 - Add a `virtual_node_peer_attributes` and `virtual_node_extra_label` arguments to `otelcol.connector.servicegraph` ([#​5058](grafana/alloy#5058)) ([20900c6](grafana/alloy@20900c6)) - Add an `otelcol.processor.metric_start_time` component ([#​5342](grafana/alloy#5342)) ([3fb13ac](grafana/alloy@3fb13ac)) - Add job level `period`, `length`, and `add_cloudwatch_timestamp` options and labels\_snake\_case to CW exporter \[backport] ([#​5355](grafana/alloy#5355)) ([60d73b7](grafana/alloy@60d73b7)) - Add missing configuration parameter `deployment_name_from_replicaset` to k8sattributes processor ([#​5183](grafana/alloy#5183)) ([b54ca77](grafana/alloy@b54ca77)) - Add parcas symbols upload to pyroscope.ebpf ([#​4948](grafana/alloy#4948)) ([30f2242](grafana/alloy@30f2242)) - Add sharding for loki.write ([#​4882](grafana/alloy#4882)) ([7570d65](grafana/alloy@7570d65)) - Add unexposed otel engine and extension to codebase and change build structure ([#​5114](grafana/alloy#5114)) ([6438176](grafana/alloy@6438176)) - **beyla.ebpf:** Add meta\_cache\_address to beyla.ebpf.attributes.kubernetes ([#​4871](grafana/alloy#4871)) ([08796f8](grafana/alloy@08796f8)) - **beyla.ebpf:** Upgrade Beyla to v2.8.5 ([f1f457f](grafana/alloy@f1f457f)) - Change the defaults for `sending_queue` > `batch` block inside `otelcol.exporter` components ([#​5061](grafana/alloy#5061)) ([714a2ed](grafana/alloy@714a2ed)) - **cluster:** Support DNS discovery mode prefixes in --cluster.join-addresses flag ([#​5034](grafana/alloy#5034)) ([08796f8](grafana/alloy@08796f8)) - **converter:** Update promtail converter to use file\_match block for loki.source.file ([#​4791](grafana/alloy#4791)) ([08796f8](grafana/alloy@08796f8)) - Database\_observability: add health check collector for postgres component ([#​5222](grafana/alloy#5222)) ([80917b1](grafana/alloy@80917b1)) - Database\_observability: expose `exclude_schemas` and `exclude_databases` settings ([#​5334](grafana/alloy#5334)) ([37656f8](grafana/alloy@37656f8)) - Database\_observability: support Azure cloud provider config data ([#​5245](grafana/alloy#5245)) ([d7a469f](grafana/alloy@d7a469f)) - Database\_observability: support Azure privatelink db name ([#​5260](grafana/alloy#5260)) ([22e4991](grafana/alloy@22e4991)) - Database\_observability.mysql: support excluding schemas in all collectors \[backport] ([#​5380](grafana/alloy#5380)) ([d67268c](grafana/alloy@d67268c)) - Database\_observability.postgres: support excluding DBs in all collectors \[backport] ([#​5383](grafana/alloy#5383)) ([165492c](grafana/alloy@165492c)) - **database\_observability:** Add health\_check collector to validate configuration ([#​5115](grafana/alloy#5115)) ([6d96740](grafana/alloy@6d96740)) - **database\_observability:** Always send explain plan log for each query including status ([#​4969](grafana/alloy#4969)) ([08796f8](grafana/alloy@08796f8)) - **database\_observability:** Append cloud provider information labels to metrics ([#​4942](grafana/alloy#4942)) ([08796f8](grafana/alloy@08796f8)) - **database\_observability:** Stop tracking own instrumentation queries ([#​4991](grafana/alloy#4991)) ([0b55557](grafana/alloy@0b55557)) - **deps:** Update Prometheus to v3.8.0 and Loki to v3.6.2 ([#​5035](grafana/alloy#5035)) ([08796f8](grafana/alloy@08796f8)) - Expose otel subcommand and add user-facing documentation ([#​5244](grafana/alloy#5244)) ([93f20b8](grafana/alloy@93f20b8)) - Improve faro.receiver.sourcemaps caching strategy ([#​4337](grafana/alloy#4337)) ([41e655c](grafana/alloy@41e655c)) - **loki.process:** Mark stage.windowsevent as GA ([#​4879](grafana/alloy#4879)) ([08796f8](grafana/alloy@08796f8)) - **loki.source.file:** Refactor tailer to reduce resource usage ([#​5003](grafana/alloy#5003)) ([08796f8](grafana/alloy@08796f8)) - **loki.source.syslog:** Implement livedebugging support ([#​5216](grafana/alloy#5216)) ([e26badb](grafana/alloy@e26badb)) - **loki.source.syslog:** Support cisco-specific syslog fields ([#​5165](grafana/alloy#5165)) ([3230ba0](grafana/alloy@3230ba0)) - **loki.source.syslog:** Support raw format ([#​5140](grafana/alloy#5140)) ([923d127](grafana/alloy@923d127)) - **mimir.alerts.kubernetes:** Add `alertmanagerconfig_matcher` block to change the matcher strategy ([f2b9671](grafana/alloy@f2b9671)) - **mimir.alerts.kubernetes:** Add component to discover AlertmanagerConfig Kubernetes resources ([#​3448](grafana/alloy#3448)) ([08796f8](grafana/alloy@08796f8)) - **mixin:** Provide rendered mixin outputs ([#​5118](grafana/alloy#5118)) ([738b9fb](grafana/alloy@738b9fb)) - **otelcol.auth.basic:** Add htpasswd file based authentication ([#​3916](grafana/alloy#3916)) ([08796f8](grafana/alloy@08796f8)) - **otelcol.connector.count:** Add component to count spans, metrics, and logs ([#​4913](grafana/alloy#4913)) ([08796f8](grafana/alloy@08796f8)) - **otelcol.exporter.file:** Add `otelcol.exporter.file` component to write metrics, logs, and traces to disk with optional rotation, compression, and grouping by resource attribute ([#​4475](grafana/alloy#4475)) ([08796f8](grafana/alloy@08796f8)) - **otelcol.exporter.prometheus:** Add `honor_metadata` config argument \[backport] ([#​5439](grafana/alloy#5439)) ([32cb175](grafana/alloy@32cb175)) - **otelcol.receiver.awss3:** Add experimental receiver for traces stored in S3 ([#​4928](grafana/alloy#4928)) ([08796f8](grafana/alloy@08796f8)) - **otelcol.receiver.kafka:** Deprecate the `topic` attribute inside the `logs`, `metrics`, and `traces` blocks in favour of a new `topics` attribute. ([f1f457f](grafana/alloy@f1f457f)) - **otelcol.receiver.kafka:** Remove the global `topic` attribute ([f1f457f](grafana/alloy@f1f457f)) - **otelcol:** Upgrade to OTel Collector v0.142.0 ([f1f457f](grafana/alloy@f1f457f)) - **prometheus.echo:** Add component for local metrics inspection in exposition format ([#​4105](grafana/alloy#4105)) ([08796f8](grafana/alloy@08796f8)) - **prometheus.exporter.cloudwatch:** Add delay option to account for CloudWatch ingestion latency ([#​4936](grafana/alloy#4936)) ([08796f8](grafana/alloy@08796f8)) - **prometheus.exporter.databricks:** Add Databricks exporter component ([#​5054](grafana/alloy#5054)) ([4442836](grafana/alloy@4442836)) - **prometheus.operator.scrapeconfigs:** Add HTTP service discovery support via httpSDConfigs ([#​4826](grafana/alloy#4826)) ([08796f8](grafana/alloy@08796f8)) - **prometheus.remote\_write:** Add metadata support to `prometheus.remote_write` component, but only if Remote Write v2 has been configured. In order for `prometheus.remote_write` to receive metadata, `prometheus.scrape` must be configured with `honor_metadata = true`. ([#​5045](grafana/alloy#5045)) ([08796f8](grafana/alloy@08796f8)) - **prometheus:** Reduce resource overhead by removing unnecessary labelstore usage ([#​4890](grafana/alloy#4890)) ([08796f8](grafana/alloy@08796f8)) - **pyroscope.ebpf:** Add `lazy_mode` argument to the `pyroscope.ebpf` to defer eBPF profiler startup until there are targets to profile ([#​4824](grafana/alloy#4824)) ([08796f8](grafana/alloy@08796f8)) - **pyroscope.enrich:** Add experimental component to enrich profiles using discovery labels ([#​4797](grafana/alloy#4797)) ([08796f8](grafana/alloy@08796f8)) - Strip comments from normalized sql text in `database_observability.postgres` ([#​5005](grafana/alloy#5005)) ([a58721a](grafana/alloy@a58721a)) - Support setting default scrape limit for prometheus.operator components ([#​5280](grafana/alloy#5280)) ([40ffe08](grafana/alloy@40ffe08)) - **tracing:** Add send\_traceparent option to enable traceparent header propagation ([#​4874](grafana/alloy#4874)) ([08796f8](grafana/alloy@08796f8)) ##### Bug Fixes 🐛 - Add support for compressed files for tail package \[backport] ([#​5415](grafana/alloy#5415)) ([311662f](grafana/alloy@311662f)) - Allow loki.source.file to read renaming lines of a deleted file before it tries to re open a new one ([#​5270](grafana/alloy#5270)) ([f8b1de8](grafana/alloy@f8b1de8)) - Compute signatures from files so that loki.source.file can handle atomic writes ([#​5143](grafana/alloy#5143)) ([3090c4a](grafana/alloy@3090c4a)) - **converter:** Fix promtail converter to limit Kubernetes discovery to same node ([#​5046](grafana/alloy#5046)) ([08796f8](grafana/alloy@08796f8)) - Database\_observability: add Azure provider relabeling rules \[backport] ([#​5382](grafana/alloy#5382)) ([c121178](grafana/alloy@c121178)) - Database\_observability: allow setting limit for mysql query\_details ([#​5314](grafana/alloy#5314)) ([085f300](grafana/alloy@085f300)) - Database\_observability: fix race in postgres query samples test ([#​5315](grafana/alloy#5315)) ([4f01753](grafana/alloy@4f01753)) - Database\_observability: grant check only require SELECT *.* on perf\_schema ([#​5294](grafana/alloy#5294)) ([490017c](grafana/alloy@490017c)) - Database\_observability: reuse cloud provider regexes ([#​5262](grafana/alloy#5262)) ([6009c54](grafana/alloy@6009c54)) - Database\_observability: update BackendXID type to int64 to better map to PG xid \[backport] ([#​5373](grafana/alloy#5373)) ([1cb4b0f](grafana/alloy@1cb4b0f)) - Database\_observability: update BackendXmin type to int64 to better map to PG BIGINT ([#​5296](grafana/alloy#5296)) ([d45ccc0](grafana/alloy@d45ccc0)) - **database\_observability.mysql:** Add setup\_actors collector to avoid tracking own queries ([#​4978](grafana/alloy#4978)) ([08796f8](grafana/alloy@08796f8)) - **database\_observability.mysql:** Replace server\_id label with hash from server\_uuid and hostname ([#​4943](grafana/alloy#4943)) ([08796f8](grafana/alloy@08796f8)) - **database\_observability.postgres:** Fix schema\_details collection for mixed case table names ([#​4872](grafana/alloy#4872)) ([08796f8](grafana/alloy@08796f8)) - **database\_observability:** Improve postgres version parsing for explain plans in database\_observability component ([#​5131](grafana/alloy#5131)) ([23c7f37](grafana/alloy@23c7f37)) - **database\_observability:** Skip explain plans which lookup individual records and return no rows ([#​5203](grafana/alloy#5203)) ([b7c7cbb](grafana/alloy@b7c7cbb)) - **deps:** Update npm dependencies ([#​5190](grafana/alloy#5190)) ([cd027e2](grafana/alloy@cd027e2)) - **docker:** Fix log corruption for multiplexed long lines ([#​4713](grafana/alloy#4713)) ([08796f8](grafana/alloy@08796f8)) - Ensure the squid exporter wrapper properly brackets ipv6 addresses ([#​5199](grafana/alloy#5199)) ([ee23162](grafana/alloy@ee23162)) - Guard pyroscope otel profiler code with unix go build tag \[backport] ([#​5360](grafana/alloy#5360)) ([b1ecdb6](grafana/alloy@b1ecdb6)) - HTTP/2 is no longer always disabled in loki.write ([#​5267](grafana/alloy#5267)) ([1c97c2d](grafana/alloy@1c97c2d)) - Invalid handling of `id` in `foreach` when using discovery components ([#​5322](grafana/alloy#5322)) ([61fe184](grafana/alloy@61fe184)), closes [#​5297](grafana/alloy#5297) - Issues when reading files using non UTF-8 encoding in loki.source.file ([#​5259](grafana/alloy#5259)) ([4740276](grafana/alloy@4740276)) - **loki.process:** Implement encoding.TextMarshaler and encoding.TextUnmarshaler for TruncateSourceType \[backport] ([#​5428](grafana/alloy#5428)) ([3585393](grafana/alloy@3585393)) - **loki.process:** Remove extraneous output stage from cri stage pipeline ([#​5002](grafana/alloy#5002)) ([08796f8](grafana/alloy@08796f8)) - **loki.source.api:** Prevent dropping request when relabel rules drop a specific stream. ([#​4834](grafana/alloy#4834)) ([08796f8](grafana/alloy@08796f8)) - **loki.source.file:** Make sure position is recorded when component exit \[backport] ([#​5418](grafana/alloy#5418)) ([64fb278](grafana/alloy@64fb278)) - **loki.source.file:** Update `tail_from_end` to properly handle file encoding \[backport] ([#​5436](grafana/alloy#5436)) ([731e8e5](grafana/alloy@731e8e5)) - **mimir.alerts.kubernetes:** Fix crash when using Kubernetes secret or configmap in AlertmanagerConfig ([#​5010](grafana/alloy#5010)) ([08796f8](grafana/alloy@08796f8)) - **mixin:** Correct invalid queries in alloy logs dashboard ([#​5123](grafana/alloy#5123)) ([ad8efd3](grafana/alloy@ad8efd3)) - Only alert on cluster drift when cluster\_name is set ([#​5181](grafana/alloy#5181)) ([8b6f056](grafana/alloy@8b6f056)) - **otelcol:** Allow configuration of force\_attempt\_http2 and default to true ([#​5050](grafana/alloy#5050)) ([08796f8](grafana/alloy@08796f8)) - Perform drain when file is deleted in tail package ([#​5139](grafana/alloy#5139)) ([2e48867](grafana/alloy@2e48867)) - Preserve meta labels in loki.source.podlogs ([#​5097](grafana/alloy#5097)) ([23d787c](grafana/alloy@23d787c)) - Prevent panic in import.git when update fails ([#​5198](grafana/alloy#5198)) ([577a591](grafana/alloy@577a591)) - **prometheus.exporter.gcp:** Preserve colons in extra\_filters filter expressions ([#​5018](grafana/alloy#5018)) ([08796f8](grafana/alloy@08796f8)) - **prometheus.operator:** Enable native histogram ingestion in internal scrape manager ([#​4750](grafana/alloy#4750)) ([08796f8](grafana/alloy@08796f8)) - **relabel:** Fix default values for source\_labels to prevent labeldrop issues ([#​5059](grafana/alloy#5059)) ([08796f8](grafana/alloy@08796f8)) - Remove Parca debug info upload from user configuration \[backport] ([#​5395](grafana/alloy#5395)) ([58eb9cc](grafana/alloy@58eb9cc)) - Revert doublestar v4 update \[backport] ([#​5435](grafana/alloy#5435)) ([0e9e615](grafana/alloy@0e9e615)) - Set content-encoding header in loki.write ([#​5346](grafana/alloy#5346)) ([ffd2bea](grafana/alloy@ffd2bea)) - Show correct fallback alloy version instead of v1.13.0 ([#​5110](grafana/alloy#5110)) ([e2e96e9](grafana/alloy@e2e96e9)) - Update to use doublestar v4 ([#​5148](grafana/alloy#5148)) ([d8f0b3e](grafana/alloy@d8f0b3e)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0Mi45NS4yIiwidXBkYXRlZEluVmVyIjoiNDIuOTUuMiIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiUmVub3ZhdGUgQm90IiwiYXV0b21hdGlvbjpib3QtYXV0aG9yZWQiLCJkZXBlbmRlbmN5LXR5cGU6Om1pbm9yIl19-->
Brief description of Pull Request
Correctly expose and wire setting to exclude schemas (mysql) and databases (postgres) from data collection. At the moment, this only applies to the
explain_planscollectors.Pull Request Details
Issue(s) fixed by this Pull Request
Notes to the Reviewer
PR Checklist