Project

General

Profile

Actions
Copy link

Bug #69107

open

mon/mon-stretch-mode-5-mons-8-osds.sh: test doesn't set the election strategy to "connectivity" before attempting to enable stretch mode

Added by Laura Flores over 1 year ago. Updated 6 months ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Laura Flores
Category:
-
Target version:
-
% Done:

0%

Source:
Backport:
squid, reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
60927
Tags (freeform):
backport_processed
Merge Commit:
2676d3ede1bb529316e2a067167a05a276c6c00e
Fixed In:
v19.3.0-6350-g2676d3ede1
Released In:
v20.2.0~1530
Upkeep Timestamp:
2025-11-01T01:01:20+00:00
Tags:
main-failures

Description

/a/teuthology-2024-11-24_20:00:15-rados-main-distro-default-smithi/8006800

2024-11-24T21:01:17.338 INFO:tasks.workunit.client.0.smithi190.stderr:+ hostname=smithi190
2024-11-24T21:01:17.338 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd crush remove smithi190
2024-11-24T21:01:17.777 INFO:tasks.workunit.client.0.smithi190.stderr:device 'smithi190' does not appear in the crush map
2024-11-24T21:01:17.795 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd getcrushmap
2024-11-24T21:01:18.193 INFO:tasks.workunit.client.0.smithi190.stderr:25
2024-11-24T21:01:18.209 INFO:tasks.workunit.client.0.smithi190.stderr:+ crushtool --decompile crushmap
2024-11-24T21:01:18.229 INFO:tasks.workunit.client.0.smithi190.stderr:+ sed 's/^# end crush map$//' crushmap.txt
2024-11-24T21:01:18.230 INFO:tasks.workunit.client.0.smithi190.stderr:+ cat
2024-11-24T21:01:18.231 INFO:tasks.workunit.client.0.smithi190.stderr:+ crushtool --compile crushmap_modified.txt -o crushmap.bin
2024-11-24T21:01:18.251 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph osd setcrushmap -i crushmap.bin
2024-11-24T21:01:20.374 INFO:tasks.workunit.client.0.smithi190.stderr:27
2024-11-24T21:01:20.398 INFO:tasks.workunit.client.0.smithi190.stderr:+ ceph mon enable_stretch_mode e stretch_rule datacenter
2024-11-24T21:01:20.771 INFO:tasks.workunit.client.0.smithi190.stderr:Error EINVAL: Monitors must use the connectivity strategy to enable stretch mode
2024-11-24T21:01:20.777 DEBUG:teuthology.orchestra.run:got remote process result: 22
2024-11-24T21:01:20.777 INFO:tasks.workunit:Stopping ['mon/mon-stretch-mode-5-mons-8-osds.sh'] on client.0...
2024-11-24T21:01:20.777 DEBUG:teuthology.orchestra.run.smithi190:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0
2024-11-24T21:01:21.137 ERROR:teuthology.run_tasks:Saw exception from tasks.

Related issues 2 (1 open — 1 closed)

Copied to RADOS - Backport #69168: squid: mon/mon-stretch-mode-5-mons-8-osds.sh: test doesn't set the election strategy to "connectivity" before attempting to enable stretch modeResolvedLaura FloresActions
Copied to RADOS - Backport #72588: reef: mon/mon-stretch-mode-5-mons-8-osds.sh: test doesn't set the election strategy to "connectivity" before attempting to enable stretch modeNewKamoltat (Junior) SirivadhnaActions
Actions
Copy link
 #1

Updated by Laura Flores over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to Laura Flores

I think this is a simple test fix.. I will test out a fix.

Actions
Copy link
 #2

Updated by Laura Flores over 1 year ago

The election strategy is randomly chosen for this type of test.

In this test, which passed, the "connectivity" election strategy happened to be picked (note the "mon_election/connectivity" bit in the description):
/a/teuthology-2024-12-01_20:00:16-rados-main-distro-default-smithi/8016518

description: rados/singleton/{all/stretch-mode-5-mons-8-osds mon_election/connectivity
    msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-stupid}
    rados supported-random-distro$/{centos_latest}}
...
   ceph:
      conf:
        global:
          mon client directed command retry: 5
          mon election default strategy: 3

In this test, which failed, the "classic" election strategy was picked:
/a/yuriw-2024-11-20_16:10:40-rados-wip-yuri2-testing-2024-11-15-0902-distro-default-smithi/8001934

description: rados/singleton/{all/stretch-mode-5-mons-8-osds mon_election/classic
    msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_random}
    bluestore/bluestore-comp-zlib} rados supported-random-distro$/{centos_latest}}
...
    ceph:
      conf:
        global:
          mon client directed command retry: 5
          mon election default strategy: 1

So we need to explicitly set the election strategy in the workunit with "ceph mon set election_strategy connectivity".

Actions
Copy link
 #3

Updated by Laura Flores over 1 year ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 60927
Actions
Copy link
 #4

Updated by Radoslaw Zarzynski over 1 year ago

scrub note: review in progres.

Actions
Copy link
 #5

Updated by Kamoltat (Junior) Sirivadhna over 1 year ago

merged, I assume we haven't seen this in Squid just yet? I guess if we ever find one, it won't be difficult to just create a backport

Actions
Copy link
 #6

Updated by Kamoltat (Junior) Sirivadhna over 1 year ago

  • Status changed from Fix Under Review to Resolved
Actions
Copy link
 #7

Updated by Laura Flores over 1 year ago

  • Status changed from Resolved to Pending Backport
  • Backport set to squid
Actions
Copy link
 #8

Updated by Upkeep Bot over 1 year ago

  • Copied to Backport #69168: squid: mon/mon-stretch-mode-5-mons-8-osds.sh: test doesn't set the election strategy to "connectivity" before attempting to enable stretch mode added
Actions
Copy link
 #9

Updated by Upkeep Bot over 1 year ago

  • Tags (freeform) set to backport_processed
Actions
Copy link
 #10

Updated by Laura Flores over 1 year ago

/a/yuriw-2024-12-03_16:16:51-rados-wip-yuri6-testing-2024-12-02-1528-distro-default-smithi/8019123

Actions
Copy link
 #11

Updated by Naveen Naidu over 1 year ago

/a/skanta-2024-12-20_03:35:31-rados-wip-bharath6-testing-2024-12-19-0956-squid-distro-default-smithi/8045473

Actions
Copy link
 #12

Updated by Laura Flores over 1 year ago

  • Status changed from Pending Backport to Resolved
  • Backport deleted (squid)

Marking resolved since there are no reports of this in Squid.

Actions
Copy link
 #13

Updated by Nitzan Mordechai over 1 year ago

/a/nmordech-2025-01-23_15:17:27-rados-wip-yuri-testing-2025-01-21-0747-squid-distro-default-smithi/8090318

Actions
Copy link
 #14

Updated by Laura Flores over 1 year ago

  • Status changed from Resolved to Pending Backport
  • Backport set to squid
Actions
Copy link
 #15

Updated by Nitzan Mordechai about 1 year ago

/yuriw-2025-02-25_18:56:42-rados-wip-yuri11-testing-2025-02-19-0801-squid-distro-default-smithi/8152067

Actions
Copy link
 #16

Updated by Laura Flores about 1 year ago

/a/yuriw-2025-02-26_00:53:37-rados-wip-yuri2-testing-2025-02-25-1331-squid-distro-default-smithi/8153912

Actions
Copy link
 #17

Updated by Shraddha Agrawal about 1 year ago

/a/yuriw-2025-03-01_16:17:18-rados-wip-yuri5-testing-2025-02-28-0824-squid-distro-default-smithi/8162995

Actions
Copy link
 #18

Updated by Naveen Naidu about 1 year ago

/a/skanta-2025-02-18_22:53:18-rados-wip-bharath2-testing-2025-02-18-1645-squid-distro-default-smithi/8138296

Actions
Copy link
 #19

Updated by Nitzan Mordechai about 1 year ago

/a/skanta-2025-03-22_15:42:53-rados-wip-bharath3-testing-2025-03-22-1505-squid-distro-default-smithi/8202818

Actions
Copy link
 #20

Updated by Laura Flores about 1 year ago

/a/yuriw-2025-03-29_14:10:41-rados-wip-yuri11-testing-2025-03-28-1517-squid-distro-default-smithi/8215847

Actions
Copy link
 #21

Updated by Jaya Prakash about 1 year ago

/a/skanta-2025-03-26_22:51:58-rados-wip-bharath11-testing-2025-03-25-2034-squid-distro-default-smithi/8211799

Actions
Copy link
 #22

Updated by Kamoltat (Junior) Sirivadhna about 1 year ago

/a/yuriw-2025-03-26_14:16:14-rados-wip-yuri8-testing-2025-03-25-0803-squid-distro-default-smithi/8210863

Actions
Copy link
 #23

Updated by Nitzan Mordechai about 1 year ago

/a/skanta-2025-03-17_14:03:12-rados-wip-bharath14-testing-2025-03-15-2152-squid-distro-default-smithi/8195345

Actions
Copy link
 #24

Updated by Shraddha Agrawal about 1 year ago

/a/skanta-2025-04-05_23:06:10-rados-wip-bharath16-testing-2025-04-05-1425-squid-distro-default-smithi/8225966

Actions
Copy link
 #25

Updated by Upkeep Bot 10 months ago

  • Merge Commit set to 2676d3ede1bb529316e2a067167a05a276c6c00e
  • Fixed In set to v19.3.0-6350-g2676d3ede1b
  • Upkeep Timestamp set to 2025-07-08T18:12:54+00:00
Actions
Copy link
 #26

Updated by Upkeep Bot 10 months ago

  • Fixed In changed from v19.3.0-6350-g2676d3ede1b to v19.3.0-6350-g2676d3ede1bb
  • Upkeep Timestamp changed from 2025-07-08T18:12:54+00:00 to 2025-07-14T15:23:02+00:00
Actions
Copy link
 #27

Updated by Upkeep Bot 10 months ago

  • Fixed In changed from v19.3.0-6350-g2676d3ede1bb to v19.3.0-6350-g2676d3ede1
  • Upkeep Timestamp changed from 2025-07-14T15:23:02+00:00 to 2025-07-14T21:08:53+00:00
Actions
Copy link
 #28

Updated by Kamoltat (Junior) Sirivadhna 9 months ago

  • Backport changed from squid to squid, reef
Actions
Copy link
 #29

Updated by Kamoltat (Junior) Sirivadhna 9 months ago

  • Copied to Backport #72588: reef: mon/mon-stretch-mode-5-mons-8-osds.sh: test doesn't set the election strategy to "connectivity" before attempting to enable stretch mode added
Actions
Copy link
 #30

Updated by Kamoltat (Junior) Sirivadhna 8 months ago

/a/skanta-2025-08-16_00:23:08-rados-wip-bharath6-testing-2025-08-15-1555-reef-distro-default-smithi/8445306

Actions
Copy link
 #31

Updated by Upkeep Bot 6 months ago

  • Released In set to v20.2.0~1530
  • Upkeep Timestamp changed from 2025-07-14T21:08:53+00:00 to 2025-11-01T01:01:20+00:00
Actions
Copy link

Also available in: Atom PDF