-
Notifications
You must be signed in to change notification settings - Fork 4k
rabbitmq-diagnostics status crashes with FunctionClauseError when vm_memory_high_watermark is unavailable #15679
Description
Describe the bug
rabbitmq-diagnostics status (and rabbitmqctl status) crash with a FunctionClauseError when the node is in maintenance mode or vm_memory_monitor is otherwise unavailable.
RabbitMQ and Erlang/OTP Versions
RabbitMQ 4.0.0-dev (main branch). Erlang/OTP 26+.
How to Reproduce
- Put a node into maintenance mode:
rabbitmqctl enable_feature_flag node_maintenance_mode - Run:
rabbitmq-diagnostics --formatter json status -n rabbit
Expected Behavior
rabbitmq-diagnostics status completes successfully, reporting that the watermark is unavailable.
Actual Behavior
The command exits with rc=70 and the following stack trace:
** (FunctionClauseError) no function clause matching in RabbitMQ.CLI.Core.Memory.formatted_watermark/1
(rabbitmqctl 4.0.0-dev) lib/rabbitmq/cli/core/memory.ex:57: RabbitMQ.CLI.Core.Memory.formatted_watermark(nil)
(rabbitmqctl 4.0.0-dev) lib/rabbitmq/cli/ctl/commands/status_command.ex:274: RabbitMQ.CLI.Ctl.Commands.StatusCommand.result_map/1
(rabbitmqctl 4.0.0-dev) lib/rabbitmq/cli/ctl/commands/status_command.ex:65: RabbitMQ.CLI.Ctl.Commands.StatusCommand.output/2
(rabbitmqctl 4.0.0-dev) lib/rabbitmqctl.ex:174: RabbitMQCtl.maybe_run_command/3
(rabbitmqctl 4.0.0-dev) lib/rabbitmqctl.ex:142: anonymous fn/5 in RabbitMQCtl.do_exec_parsed_command/5
(rabbitmqctl 4.0.0-dev) lib/rabbitmqctl.ex:642: RabbitMQCtl.maybe_with_distribution/3
(rabbitmqctl 4.0.0-dev) lib/rabbitmqctl.ex:107: RabbitMQCtl.exec_command/2
(rabbitmqctl 4.0.0-dev) lib/rabbitmqctl.ex:41: RabbitMQCtl.main/1
Root Cause
When a node is in maintenance mode, the vm_memory_monitor GenServer is not running. rabbit_misc:filter_exit_map/2 in rabbit.erl silently drops the :vm_memory_high_watermark key from the status result when the call exits. Consequently, Keyword.get/2 returns nil, and nil |> Memory.formatted_watermark/1 finds no matching clause.
This is the same crash location as issue #2964, but with nil instead of {:relative, float}.
Additional Context
The text formatter (readable_watermark_setting in status_command.ex) is affected by the same issue: none of its case branches handle nil.