Project

General

Profile

Actions
Copy link

Bug #66829

open

quincy,reef: Error EINVAL: unrecognized config option 'mds_log_trim_decay_rate'

Added by Xiubo Li over 1 year ago. Updated 9 months ago.

Status:
New
Priority:
Normal
Assignee:
Rishabh Dave
Category:
Testing
Target version:
-
% Done:

0%

Source:
Q/A
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
Labels (FS):
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
Released In:
Upkeep Timestamp:

Description

https://pulpito.ceph.com/jcollin-2024-06-26_02:20:03-fs-wip-jcollin-testing-20240625.103903-reef-distro-default-smithi/7772702

2024-06-29T03:28:18.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-06-29T03:28:18.292+0000 7fa9343b0640  1 -- 172.21.15.83:0/794654387 --> [v2:172.21.15.83:3300/0,v1:172.21.15.83:6789/0] -- mon_command([{prefix=config set, name=mds_log_trim_decay_rate}] v 0) v1 -- 0x7fa92c1f0e70 con 0x7fa92c0ce4b0
2024-06-29T03:28:18.293 INFO:teuthology.orchestra.run.smithi083.stderr:2024-06-29T03:28:18.293+0000 7fa922ffd640  1 -- 172.21.15.83:0/794654387 <== mon.0 v2:172.21.15.83:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_log_trim_decay_rate}]=-22 unrecognized config option 'mds_log_trim_decay_rate' v79)=-22 unrecognized config option 'mds_log_trim_decay_rate' v79) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa92c1f0e70 con 0x7fa92c0ce4b0
2024-06-29T03:28:18.293 INFO:teuthology.orchestra.run.smithi083.stderr:Error EINVAL: unrecognized config option 'mds_log_trim_decay_rate'
2024-06-29T03:28:18.297 DEBUG:teuthology.orchestra.run:got remote process result: 22
2024-06-29T03:28:18.298 INFO:teuthology.nuke:Clearing teuthology firewall rules...
...
2024-06-29T03:28:28.754 INFO:teuthology.orchestra.run.smithi083.stderr:2024-06-29T03:28:28.751+0000 7fdd56478640  1 -- 172.21.15.83:0/2266879772 wait complete.
2024-06-29T03:28:28.771 INFO:tasks.cephfs_test_runner:Test that, when health warning MDS_TRIM is present for an MDS, command ... ERROR
2024-06-29T03:28:28.771 INFO:tasks.cephfs_test_runner:
2024-06-29T03:28:28.771 INFO:tasks.cephfs_test_runner:======================================================================
2024-06-29T03:28:28.771 INFO:tasks.cephfs_test_runner:ERROR: test_with_health_warn_trim (tasks.cephfs.test_admin.TestFSFail)
2024-06-29T03:28:28.771 INFO:tasks.cephfs_test_runner:Test that, when health warning MDS_TRIM is present for an MDS, command
2024-06-29T03:28:28.771 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2024-06-29T03:28:28.771 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_570cd3094b5bed39a1caaa3ff315a0622e9d03e6/qa/tasks/cephfs/test_admin.py", line 1791, in test_with_health_warn_trim
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:    self.config_set('mds', 'mds_log_trim_decay_rate', '60')
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_570cd3094b5bed39a1caaa3ff315a0622e9d03e6/qa/tasks/ceph_test_case.py", line 203, in config_set
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:    self.ceph_cluster.mon_manager.raw_cluster_cmd("config", "set", section, key, str(value))
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_570cd3094b5bed39a1caaa3ff315a0622e9d03e6/qa/tasks/ceph_manager.py", line 1616, in raw_cluster_cmd
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:    return self.run_cluster_cmd(**kwargs).stdout.getvalue()
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_570cd3094b5bed39a1caaa3ff315a0622e9d03e6/qa/tasks/ceph_manager.py", line 1607, in run_cluster_cmd
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:    return self.controller.run(**kwargs)
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/remote.py", line 523, in run
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 455, in run
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:    r.wait()
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 161, in wait
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:    self._raise_for_status()
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/git.ceph.com_teuthology_544fecbcd55f3d2b6f478823823ce40cbefef1d4/teuthology/orchestra/run.py", line 181, in _raise_for_status
2024-06-29T03:28:28.772 INFO:tasks.cephfs_test_runner:    raise CommandFailedError(
2024-06-29T03:28:28.773 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi083 with status 22: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_log_trim_decay_rate 60'
2024-06-29T03:28:28.773 INFO:tasks.cephfs_test_runner:
2024-06-29T03:28:28.773 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2024-06-29T03:28:28.773 INFO:tasks.cephfs_test_runner:Ran 29 tests in 1235.420s
2024-06-29T03:28:28.773 INFO:tasks.cephfs_test_runner:
2024-06-29T03:28:28.773 INFO:tasks.cephfs_test_runner:FAILED (errors=1)
Actions
Copy link
 #1

Updated by Xiubo Li over 1 year ago

  • Subject changed from Error EINVAL: unrecognized config option 'mds_log_trim_decay_rate' to reef: Error EINVAL: unrecognized config option 'mds_log_trim_decay_rate'
  • Backport deleted (quincy,reef,squid)
Actions
Copy link
 #2

Updated by Xiubo Li over 1 year ago ยท Edited

Introduced by https://github.com/ceph/ceph/pull/57837 and the following commit is missing:

commit 29610577eece04c028c412f112a66fafa8f70316
Author: Venky Shankar <vshankar@redhat.com>
Date:   Mon Jul 24 00:33:47 2023 -0400

    mds: add mdlog trimming threshold and decay counter

    Fixes: http://tracker.ceph.com/issues/61908
    Signed-off-by: Venky Shankar <vshankar@redhat.com>
Actions
Copy link
 #3

Updated by Venky Shankar over 1 year ago

  • Category set to Testing
  • Assignee set to Rishabh Dave
  • Target version set to v19.1.1
  • Source set to Q/A

Xiubo Li wrote in #note-2:

Introduced by https://github.com/ceph/ceph/pull/57837 and the following commit is missing:

[...]

That's because we deliberately chose to not backport this change to reef (and quincy). The failing tests (test_admin.py) should use old configs (in reef/quincy) to slow down trimming as per test requirements.

Actions
Copy link
 #7

Updated by Venky Shankar over 1 year ago

This also shows up in quincy: https://pulpito.ceph.com/xiubli-2024-08-13_04:48:23-fs-wip-jcollin-testing-20240812.053224-quincy-distro-default-smithi/7850991/

@Rishabh Dave - I do recall a PR for this. Could you link ti in the tracker please?

Actions
Copy link
 #8

Updated by Venky Shankar over 1 year ago

  • Subject changed from reef: Error EINVAL: unrecognized config option 'mds_log_trim_decay_rate' to quincy,reef: Error EINVAL: unrecognized config option 'mds_log_trim_decay_rate'
Actions
Copy link
 #9

Updated by Ilya Dryomov about 1 year ago

  • Target version deleted (v19.1.1)
Actions
Copy link

Also available in: Atom PDF