Skip to content

Conversation

@IvanHunters
Copy link
Collaborator

@IvanHunters IvanHunters commented Dec 10, 2025

[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant monitoring
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the Alerta deployment to enable
filtering of alert severities for Slack notifications based on the disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics within tenant namespaces, improving
monitoring granularity and control.

[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.

Summary by CodeRabbit

  • New Features

    • Added configurable severity filtering for Telegram alerts.
    • Extended Slack severity filtering to accept lists of severities.
  • Bug Fixes / Behavior

    • Severity settings now accept arrays (multiple severities) instead of single comma-separated strings.
  • Documentation

    • Updated configuration docs and examples to show list-style severity settings.

✏️ Tip: You can customize this high-level summary in your review settings.

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Dec 10, 2025
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Dec 10, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Introduce array-based disabledSeverity fields for Slack and Telegram and emit JSON-quoted severity filters as environment variables in the Alerta deployment template when corresponding integrations are enabled.

Changes

Cohort / File(s) Summary
Alerta deployment template
packages/extra/monitoring/templates/alerta/alerta.yaml
Conditionally emit two new env vars: TELEGRAM_DISABLE_NOTIFICATION_SEVERITY from .Values.alerta.alerts.telegram.disabledSeverity (JSON-quoted) when Telegram is configured, and SLACK_SEVERITY_FILTER from .Values.alerta.alerts.slack.disabledSeverity (JSON-quoted) when Slack is configured and severity is provided.
Schema & chart values
packages/extra/monitoring/values.schema.json, packages/extra/monitoring/values.yaml
Change disabledSeverity for Telegram from string to array of strings; add disabledSeverity as array of strings for Slack. Defaults set to empty arrays and descriptions updated to show array examples.
Documentation
packages/extra/monitoring/README.md
Update Alerta configuration table formatting and reflect disabledSeverity fields as arrays with example values.
Cluster resource schema / keys order
packages/system/cozystack-resource-definitions/cozyrds/monitoring.yaml
Update embedded OpenAPI schema and chart values JSON to use array disabledSeverity for Slack and Telegram; add new keysOrder entries for the Slack/Grafana disabledSeverity paths.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Review template conditionals and correct JSON quoting/escaping for env values in alerta.yaml.
  • Confirm schema change (string → array) is consistent across values.schema.json, values.yaml, and embedded OpenAPI in monitoring.yaml.
  • Validate README examples match the new types.
  • Check keysOrder modifications in monitoring.yaml for ordering correctness.

Poem

I’m a rabbit in the config glade, hopping light and merry,
I changed severities from lone strings to a boundless array,
Slack and Telegram now wear tidy filters like a crown,
I nibble bugs and stitch the YAML gown,
Hooray for clearer alerts — carrots for the deploy day! 🥕🐇

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Title check ⚠️ Warning The title mentions SLACK_SEVERITY_FILTER and VMAgent monitoring, but the primary changes are converting disabledSeverity from string to array types across Telegram and Slack configurations, with template updates to use these new arrays. Revise the title to accurately reflect the main change: something like '[monitoring] Convert disabledSeverity to array type for Slack and Telegram alerts' would better represent the core modifications.
✅ Passed checks (2 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch feature/slack-notification-from-alerta

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@dosubot dosubot bot added the enhancement New feature or request label Dec 10, 2025
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @IvanHunters, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the monitoring-agents package by integrating Alertmanager, a crucial component for handling and routing alerts. The changes establish a complete alert notification pipeline, primarily focusing on Slack integration, and provide the necessary Kubernetes resources and configuration defaults to deploy and manage Alertmanager effectively. This ensures that critical system alerts can be efficiently processed and dispatched to relevant communication channels.

Highlights

  • Alertmanager Integration: Introduced a new Alertmanager deployment, service, and configuration secret to enable robust alert routing and notification capabilities within the monitoring system.
  • Slack Notification Setup: Configured Alertmanager to send notifications to Slack, utilizing templated values for webhook URL, channel, username, and icon emoji, making it easily configurable.
  • VMAlert Integration: Added a VMAlert custom resource that is configured to forward alerts to the newly deployed Alertmanager instance.
  • Workload Monitoring: Included a WorkloadMonitor custom resource specifically for Alertmanager, defining its monitoring characteristics within the system.
  • Configuration Defaults: Added default configuration values for Alertmanager in values.yaml, allowing for easy enablement, replica scaling, and Slack integration setup.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces Alertmanager with Slack integration to the monitoring stack. While this is a valuable addition, the current implementation has several areas that need improvement to be considered production-ready. My review includes critical feedback on using a Deployment for a stateful service like Alertmanager, which should be a StatefulSet to handle state and high availability correctly. I've also pointed out issues with hardcoded values, conditional configuration for the Slack integration to prevent errors, and missing resource definitions. Addressing these points will significantly improve the robustness and maintainability of this new component.

@@ -0,0 +1,54 @@
{{- if .Values.alertmanager.enabled }}
apiVersion: apps/v1
kind: Deployment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

For a stateful application like Alertmanager, a StatefulSet is more appropriate than a Deployment. A StatefulSet provides stable network identifiers and persistent storage, which are crucial for features like alert deduplication, silences, and high availability.

Using a Deployment with an emptyDir volume (as configured on line 53) will cause all state to be lost on pod restarts. Furthermore, disabling clustering via --cluster.listen-address= (line 26) will lead to duplicate notifications if you scale to more than one replica.

Comment on lines 18 to 38
route:
group_by: ['alertname']
group_wait: 10s
group_interval: 10s
repeat_interval: 1h
receiver: 'slack'
routes:
- match:
alertname: Watchdog
receiver: 'null'
receivers:
- name: 'null'
- name: 'slack'
slack_configs:
- api_url: {{ .Values.alertmanager.slack.webhookUrl | quote }}
channel: {{ .Values.alertmanager.slack.channel | quote }}
username: {{ .Values.alertmanager.slack.username | quote }}
icon_emoji: {{ .Values.alertmanager.slack.iconEmoji | quote }}
title: '{{`{{ .GroupLabels.alertname }}`}}'
text: '{{`{{ range .Alerts }}{{ .Annotations.summary }} - {{ .Annotations.description }}{{ end }}`}}'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The configuration unconditionally sets 'slack' as the default receiver and defines the slack_configs. If alertmanager.slack.webhookUrl is not provided in values.yaml, Alertmanager will fail to send notifications because the api_url will be empty.

To prevent this, you should conditionally configure the receiver and the route based on the presence of the webhookUrl.

    route:
      group_by: ['alertname']
      group_wait: 10s
      group_interval: 10s
      repeat_interval: 1h
      receiver: {{ if .Values.alertmanager.slack.webhookUrl }}'slack'{{ else }}'null'{{ end }}
      routes:
      - match:
          alertname: Watchdog
        receiver: 'null'

    receivers:
    - name: 'null'
    {{- if .Values.alertmanager.slack.webhookUrl }}
    - name: 'slack'
      slack_configs:
      - api_url: {{ .Values.alertmanager.slack.webhookUrl | quote }}
        channel: {{ .Values.alertmanager.slack.channel | quote }}
        username: {{ .Values.alertmanager.slack.username | quote }}
        icon_emoji: {{ .Values.alertmanager.slack.iconEmoji | quote }}
        title: '{{`{{ .GroupLabels.alertname }}`}}'
        text: '{{`{{ range .Alerts }}{{ .Annotations.summary }} - {{ .Annotations.description }}{{ end }}`}}'
    {{- end }}

Comment on lines 20 to 21
image: "prom/alertmanager:v0.27.0"
imagePullPolicy: IfNotPresent
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The Alertmanager image, tag, and pull policy are hardcoded. It's a best practice to make these configurable in values.yaml to simplify upgrades.

Please add an image object to the alertmanager section in values.yaml like this:

alertmanager:
  # ... other values
  image:
    repository: prom/alertmanager
    tag: v0.27.0
    pullPolicy: IfNotPresent

Then use these values in the template.

          image: "{{ .Values.alertmanager.image.repository }}:{{ .Values.alertmanager.image.tag }}"
          imagePullPolicy: {{ .Values.alertmanager.image.pullPolicy }}

name: alertmanager
spec:
replicas: {{ .Values.alertmanager.replicas }}
minReplicas: 1
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

minReplicas is hardcoded to 1. If a user sets alertmanager.replicas to 0 in values.yaml to disable the deployment, this WorkloadMonitor will have replicas: 0 and minReplicas: 1, which is an impossible condition to satisfy and will likely result in a persistent non-operational status for this workload. minReplicas should be 0 when the desired replica count is 0.

  minReplicas: {{ if gt (int .Values.alertmanager.replicas) 0 }}1{{ else }}0{{ end }}

url: http://vmselect-shortterm.tenant-root.svc:8481/select/0/prometheus
remoteWrite:
url: http://vminsert-shortterm.tenant-root.svc:8480/insert/0/prometheus/api/v1/write
resources: {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The resources for VMAlert are empty. It is a best practice to specify resource requests and limits for all workloads to ensure cluster stability and proper scheduling. These should be configurable via values.yaml.

Example for values.yaml:

vmalert:
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 200m
      memory: 256Mi
  resources: {{- toYaml .Values.vmalert.resources | nindent 4 }}

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 10d3574 and 8ca7e80.

📒 Files selected for processing (6)
  • packages/system/monitoring-agents/templates/alertmanager-deployment.yaml (1 hunks)
  • packages/system/monitoring-agents/templates/alertmanager-secret.yaml (1 hunks)
  • packages/system/monitoring-agents/templates/alertmanager-service.yaml (1 hunks)
  • packages/system/monitoring-agents/templates/alertmanager-workloadmonitor.yaml (1 hunks)
  • packages/system/monitoring-agents/templates/vmalert.yaml (1 hunks)
  • packages/system/monitoring-agents/values.yaml (1 hunks)
🧰 Additional context used
🪛 YAMLlint (1.37.1)
packages/system/monitoring-agents/templates/alertmanager-secret.yaml

[error] 1-1: syntax error: expected the node content, but found '-'

(syntax)

packages/system/monitoring-agents/templates/alertmanager-deployment.yaml

[error] 1-1: syntax error: expected the node content, but found '-'

(syntax)

packages/system/monitoring-agents/templates/alertmanager-workloadmonitor.yaml

[error] 1-1: syntax error: expected the node content, but found '-'

(syntax)

packages/system/monitoring-agents/templates/alertmanager-service.yaml

[error] 1-1: syntax error: expected the node content, but found '-'

(syntax)

packages/system/monitoring-agents/templates/vmalert.yaml

[error] 1-1: syntax error: expected the node content, but found '-'

(syntax)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Build
🔇 Additional comments (8)
packages/system/monitoring-agents/values.yaml (1)

424-438: Alert manager configuration defaults look reasonable.

The feature is opt-in (disabled by default), resource limits are conservative, and Slack webhook is empty (requires user configuration to activate). Values are well-structured.

packages/system/monitoring-agents/templates/alertmanager-deployment.yaml (1)

23-26: Verify clustering behavior if replicas are scaled beyond 1.

Line 26 disables clustering with --cluster.listen-address= (empty value). This works correctly for the default replicas: 1, but if someone increases replicas, clustering will remain disabled, potentially causing alert deduplication and routing failures across multiple instances. Consider either:

  1. Making clustering conditional based on replica count, or
  2. Adding documentation warning users not to scale replicas without enabling clustering

Is single-replica-only the intended design, or should multi-replica deployments support clustering?

packages/system/monitoring-agents/templates/alertmanager-service.yaml (1)

1-17: Service definition is correct. YAMLlint error is a false positive.

The Service correctly exposes Alertmanager on port 9093 with proper selector. The YAMLlint error about "syntax error: expected the node content, but found '-'" is a false positive—Helm's {{- syntax is valid and removes whitespace. This can be safely ignored.

packages/system/monitoring-agents/templates/alertmanager-secret.yaml (2)

11-16: SMTP defaults are placeholder values and will not function.

Lines 12-16 configure SMTP to localhost:587 with empty credentials and no TLS. These are placeholder defaults that won't work for actual email notifications. This is likely intentional for an MVP, but users enabling Alertmanager should understand that SMTP requires explicit configuration outside of these values.

Verify that documentation exists explaining how to configure SMTP credentials if users want email notifications (or confirm SMTP is out of scope for this PR).


29-38: Slack configuration is properly templated and safe.

Webhook URL defaults to empty, preventing accidental misconfiguration. Channel, username, and iconEmoji have sensible defaults and are configurable via values. Alert templates correctly use Helm's {{...}} escaping to render Alertmanager template variables (alertname, summary, description).

packages/system/monitoring-agents/templates/alertmanager-workloadmonitor.yaml (1)

1-14: WorkloadMonitor definition is correct.

The selector matches the Alertmanager deployment labels, replicas are sourced from values for consistency, and minReplicas constraint is reasonable. No issues.

packages/system/monitoring-agents/templates/vmalert.yaml (2)

7-8: Hardcoded service URLs limit flexibility and assume specific deployment topology.

Lines 7-8 and 12-15 hardcode VMAlert datasource and write URLs to vmselect-shortterm and vminsert-shortterm services. These URLs are not configurable from values and assume a specific cozystack deployment topology. If the monitoring stack architecture changes, these URLs will break silently.

Consider moving these URLs to values.yaml as configurable parameters so deployments can adapt to different topologies.

Verify that these hardcoded service names and paths are correct and stable for cozystack deployments. If topology is expected to vary, these should be configurable.

Also applies to: 12-15


17-17: Verify that selectAllByDefault: true is intentional.

This setting causes VMAlert to evaluate ALL PrometheusRules in the cluster. While this may be desired, it's very broad and could lead to unexpected rule evaluation or high resource load. Confirm this is the intended behavior, or consider scoping rules by namespace/labels for better control.

Is selecting all PrometheusRules cluster-wide the expected behavior, or should rule evaluation be scoped (e.g., to specific namespaces or with label selectors)?

url: http://vmselect-shortterm.tenant-root.svc:8481/select/0/prometheus
remoteWrite:
url: http://vminsert-shortterm.tenant-root.svc:8480/insert/0/prometheus/api/v1/write
resources: {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

VMAlert has no resource requests or limits.

Line 16 sets resources: {}, which means VMAlert pod has no guaranteed memory/CPU and no upper limits. This could lead to resource contention or out-of-memory kills. Provide sensible defaults (similar to Alertmanager's 100m/128Mi request) or make it configurable from values.

Apply this diff to provide reasonable resource defaults:

-  resources: {}
+  resources:
+    requests:
+      cpu: 100m
+      memory: 128Mi
+    limits:
+      cpu: 500m
+      memory: 512Mi

Adjust these values based on your evaluation interval and rule complexity, or expose them in values.yaml for flexibility.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
resources: {}
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 512Mi
🤖 Prompt for AI Agents
In packages/system/monitoring-agents/templates/vmalert.yaml around line 16, the
VMAlert manifest currently has resources: {} which provides no CPU/memory
requests or limits; update the template to set sensible default resource
requests and limits (e.g., requests: cpu 100m, memory 128Mi and limits: cpu
250m, memory 256Mi) and make these values configurable via values.yaml (use Helm
template placeholders for values.alertmanager.resources.* with the defaults), so
operators can override them without changing the chart.

Copy link
Member

@kvaps kvaps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will route all alerts bypassing alerta, which is IRM and we already have Slack configuration for it.

All you need is to add SLACK_SEVERITY_FILTER enviroment variable for slack that works the similar way like TELEGRAM_DISABLE_NOTIFICATION_SEVERITY enviroment variable

Please RTFS

https://github.com/alerta/alerta-contrib/blob/39d64e1b255689ca386b5ecd6be75642543693ec/plugins/telegram/alerta_telegram.py#L117-L119

https://github.com/alerta/alerta-contrib/blob/39d64e1b255689ca386b5ecd6be75642543693ec/plugins/slack/alerta_slack.py#L208-L211

@IvanHunters IvanHunters force-pushed the feature/slack-notification-from-alerta branch from 8ca7e80 to 3973af2 Compare December 11, 2025 20:50
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Dec 11, 2025
@IvanHunters IvanHunters force-pushed the feature/slack-notification-from-alerta branch from 3973af2 to a632194 Compare December 11, 2025 20:52
@dosubot dosubot bot added size:XS This PR changes 0-9 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Dec 11, 2025
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8ca7e80 and a632194.

📒 Files selected for processing (3)
  • packages/extra/monitoring/templates/alerta/alerta.yaml (1 hunks)
  • packages/extra/monitoring/values.schema.json (1 hunks)
  • packages/extra/monitoring/values.yaml (2 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Build
🔇 Additional comments (3)
packages/extra/monitoring/values.schema.json (1)

28-32: Schema definition looks good.

The severityFilter property is properly defined with appropriate type, default, and documentation. It's correctly marked as optional (not in the required array), and the example format is clear.

packages/extra/monitoring/values.yaml (2)

104-106: Documentation is clear and consistent.

The JSDoc typedef properly documents the new severityFilter optional field with a helpful example. This aligns well with the schema definition and provides good guidance to users configuring Slack alerts.


134-136: Default configuration value is sensible.

Setting severityFilter to an empty string provides a good default that allows opt-in filtering without breaking existing configurations. The placement within the slack alert configuration block is correct.

Comment on lines 136 to 145
{{- if .Values.alerta.alerts.slack.url }}
- name: "SLACK_WEBHOOK_URL"
value: "{{ .Values.alerta.alerts.slack.url }}"
- name: SLACK_SEVERITY_FILTER
value: "{{ .Values.alerta.alerts.slack.severityFilter }}"
{{- end }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🌐 Web query:

Alerta 9.0.4 Slack plugin SLACK_SEVERITY_FILTER environment variable

💡 Result:

Do you want (A) what SLACK_SEVERITY_FILTER does and allowed values, (B) how to set it for Alerta 9.0.4, or (C) help locating it in the Slack plugin source? I can fetch the exact plugin code and examples once you confirm.

References I can look at: Alerta 9.0 docs (configuration/plugins) and the alerta-contrib plugins repo. [1][2]

[1] https://docs.alerta.io/configuration.html
[2] https://github.com/alerta/alerta-contrib (plugins list)


🌐 Web query:

Alerta Slack alert plugin documentation environment variables

💡 Result:

Short answer — where to find it and how to proceed:

  • The Slack notification plugin for Alerta is provided in the alerta-contrib repository (plugins/slack). See the repo. [1]
  • Alerta plugin enabling and environment-variable behaviour are documented in the main Alerta docs: enable the plugin by adding "slack" to PLUGINS in your alertad.conf (or PLUGINS env var) and note that environment variables override config file settings. [2][3]
  • If you need the exact plugin-specific environment variables (webhook URL, channel, username, etc.), they are defined by the Slack plugin code/README in alerta-contrib — I can fetch and list the exact env var names used by that plugin (or paste the relevant lines from slack.py). Would you like me to pull those exact variables now?

Sources:

  1. alerta-contrib plugins (slack). [GitHub alerta/alerta-contrib — plugins/slack]. (github.com)
  2. Alerta configuration / plugins documentation (how to enable plugins). (docs.alerta.io)
  3. Docker image notes showing PLUGINS env var usage. (github.com)

Citations:


🏁 Script executed:

# Search for Alerta repository and Slack plugin information
curl -s https://api.github.com/repos/alerta/alerta/contents/alerta/plugins | head -100

Repository: cozystack/cozystack

Length of output: 5607


🏁 Script executed:

# Fetch the Slack plugin source from alerta-contrib
curl -s https://raw.githubusercontent.com/alerta/alerta-contrib/master/plugins/slack/alerta_slack.py | head -150

Repository: cozystack/cozystack

Length of output: 5908


🏁 Script executed:

# Also check for README or documentation in the slack plugin directory
curl -s https://raw.githubusercontent.com/alerta/alerta-contrib/master/plugins/slack/README.md

Repository: cozystack/cozystack

Length of output: 5569


Fix SLACK_SEVERITY_FILTER environment variable format and default handling.

The SLACK_SEVERITY_FILTER environment variable is supported in the Alerta Slack plugin, but there are two issues with the current implementation:

  1. Format incompatibility: The plugin uses ast.literal_eval() to parse the environment variable, expecting Python literal syntax (e.g., ['critical', 'major']). An empty string default will fail to parse and silently fall back to an empty list [], which disables filtering entirely.

  2. Missing documentation: The variable is not documented in the Slack plugin README, creating ambiguity about expected format and behavior.

Update .Values.alerta.alerts.slack.severityFilter to either:

  • Use a valid Python list literal format (e.g., ['critical', 'major']) instead of an empty string, or
  • Remove the environment variable configuration if severity filtering is not needed
- name: SLACK_SEVERITY_FILTER
  value: "{{ .Values.alerta.alerts.slack.severityFilter }}"
🤖 Prompt for AI Agents
In packages/extra/monitoring/templates/alerta/alerta.yaml around lines 136 to
141, the SLACK_SEVERITY_FILTER env var is being set to whatever
.Values.alerta.alerts.slack.severityFilter provides, but the Alerta Slack plugin
expects a Python literal list (e.g. ['critical','major']) and will fail or
silently fallback if given an empty string; update the template to either (a)
only emit the SLACK_SEVERITY_FILTER env var when
.Values.alerta.alerts.slack.severityFilter is a non-empty, properly formatted
Python list literal, or (b) remove the env var output when no severity filter is
desired, and update the chart README to document the expected Python list
literal format and that omitting the variable disables filtering.

…onitoring

What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the Alerta deployment to enable
filtering of alert severities for Slack notifications based on the disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics within tenant namespaces, improving
monitoring granularity and control.

```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```

Signed-off-by: IvanHunters <xorokhotnikov@gmail.com>
@IvanHunters IvanHunters force-pushed the feature/slack-notification-from-alerta branch from a632194 to df3a409 Compare December 17, 2025 09:42
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Dec 17, 2025
@IvanHunters IvanHunters changed the title add alertmanager for slack configutation [monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant monitoring Dec 17, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Dec 17, 2025
Copy link
Member

@kvaps kvaps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kvaps kvaps added the backport Should change be backported on previus release label Dec 18, 2025
@kvaps kvaps merged commit 1f0f14c into main Dec 18, 2025
28 checks passed
@kvaps kvaps deleted the feature/slack-notification-from-alerta branch December 18, 2025 08:13
@github-actions
Copy link

Backport failed for release-0.38, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release-0.38
git worktree add -d .worktree/backport-1712-to-release-0.38 origin/release-0.38
cd .worktree/backport-1712-to-release-0.38
git switch --create backport-1712-to-release-0.38
git cherry-pick -x df3a4091420e9ee056beedea7e9f2892dee0e054

kvaps added a commit that referenced this pull request Dec 23, 2025
…onitoring (#1712)

[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant
monitoring
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the
Alerta deployment to enable
filtering of alert severities for Slack notifications based on the
disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics
within tenant namespaces, improving
monitoring granularity and control.

```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

* **New Features**
  * Added configurable severity filtering for Telegram alerts.
  * Extended Slack severity filtering to accept lists of severities.

* **Bug Fixes / Behavior**
* Severity settings now accept arrays (multiple severities) instead of
single comma-separated strings.

* **Documentation**
* Updated configuration docs and examples to show list-style severity
settings.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
kvaps added a commit that referenced this pull request Jan 8, 2026
…onitoring (#1712)

[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant
monitoring
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the
Alerta deployment to enable
filtering of alert severities for Slack notifications based on the
disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics
within tenant namespaces, improving
monitoring granularity and control.

```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
  * Added configurable severity filtering for Telegram alerts.
  * Extended Slack severity filtering to accept lists of severities.

* **Bug Fixes / Behavior**
* Severity settings now accept arrays (multiple severities) instead of
single comma-separated strings.

* **Documentation**
* Updated configuration docs and examples to show list-style severity
settings.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
kvaps added a commit that referenced this pull request Jan 8, 2026
…onitoring (#1712)

[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant
monitoring
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the
Alerta deployment to enable
filtering of alert severities for Slack notifications based on the
disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics
within tenant namespaces, improving
monitoring granularity and control.

```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
  * Added configurable severity filtering for Telegram alerts.
  * Extended Slack severity filtering to accept lists of severities.

* **Bug Fixes / Behavior**
* Severity settings now accept arrays (multiple severities) instead of
single comma-separated strings.

* **Documentation**
* Updated configuration docs and examples to show list-style severity
settings.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
nbykov0 pushed a commit that referenced this pull request Jan 9, 2026
…onitoring (#1712)

[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant
monitoring
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the
Alerta deployment to enable
filtering of alert severities for Slack notifications based on the
disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics
within tenant namespaces, improving
monitoring granularity and control.

```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

* **New Features**
  * Added configurable severity filtering for Telegram alerts.
  * Extended Slack severity filtering to accept lists of severities.

* **Bug Fixes / Behavior**
* Severity settings now accept arrays (multiple severities) instead of
single comma-separated strings.

* **Documentation**
* Updated configuration docs and examples to show list-style severity
settings.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
kvaps added a commit that referenced this pull request Jan 9, 2026
…onitoring (#1712)

[monitoring] Add SLACK_SEVERITY_FILTER field and VMAgent for tenant
monitoring
What this PR does
This PR introduces the SLACK_SEVERITY_FILTER environment variable in the
Alerta deployment to enable
filtering of alert severities for Slack notifications based on the
disabledSeverity configuration.
Additionally, it adds a VMAgent resource template for scraping metrics
within tenant namespaces, improving
monitoring granularity and control.

```release-note
[monitoring] Add SLACK_SEVERITY_FILTER for filtering Slack alert severities and VMAgent configuration for
tenant-specific metrics scraping.
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
  * Added configurable severity filtering for Telegram alerts.
  * Extended Slack severity filtering to accept lists of severities.

* **Bug Fixes / Behavior**
* Severity settings now accept arrays (multiple severities) instead of
single comma-separated strings.

* **Documentation**
* Updated configuration docs and examples to show list-style severity
settings.

<sub>✏️ Tip: You can customize this high-level summary in your review
settings.</sub>

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport Should change be backported on previus release enhancement New feature or request lgtm This PR has been approved by a maintainer size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants