Project

General

Profile

Actions

Bug #74517

open

drop.ceph.com is unreachable (ceph-post-file)

Added by Laura Flores about 2 months ago. Updated 2 days ago.

Status:
In Progress
Priority:
Normal
Assignee:
% Done:

0%

Source:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Tags (freeform):

Description

/a/lflores-2026-01-21_20:56:39-rados-main-distro-default-trial/11859

2026-01-22T00:37:37.599 INFO:tasks.workunit:Running workunit post-file.sh...
2026-01-22T00:37:37.599 DEBUG:teuthology.orchestra.run.trial167:workunit test post-file.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=627d6cb11df8056dbdf27d04fd5d9340a2368826 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/post-file.sh
2026-01-22T00:37:37.646 INFO:tasks.workunit.client.0.trial167.stderr:+ what=
2026-01-22T00:37:37.646 INFO:tasks.workunit.client.0.trial167.stderr:+ '[' -z '' ']'
2026-01-22T00:37:37.646 INFO:tasks.workunit.client.0.trial167.stderr:+ what=/etc/udev/rules.d
2026-01-22T00:37:37.646 INFO:tasks.workunit.client.0.trial167.stderr:+ sudo ceph-post-file -d ceph-test-workunit /etc/udev/rules.d
2026-01-22T00:37:37.654 INFO:tasks.workunit.client.0.trial167.stdout:args: -d ceph-test-workunit -- /etc/udev/rules.d
2026-01-22T00:37:37.654 INFO:tasks.workunit.client.0.trial167.stdout:args: -- /etc/udev/rules.d
2026-01-22T00:37:37.655 INFO:tasks.workunit.client.0.trial167.stdout:/usr/bin/ceph-post-file: upload tag 152ca85d-778b-49f5-b7e1-5aa0618223c5
2026-01-22T00:37:37.660 INFO:tasks.workunit.client.0.trial167.stdout:/usr/bin/ceph-post-file: user: root@trial167.maas
2026-01-22T00:37:37.661 INFO:tasks.workunit.client.0.trial167.stdout:/usr/bin/ceph-post-file: description: ceph-test-workunit
2026-01-22T00:37:37.662 INFO:tasks.workunit.client.0.trial167.stdout:/usr/bin/ceph-post-file: will upload directory /etc/udev/rules.d
2026-01-22T00:39:48.461 INFO:tasks.workunit.client.0.trial167.stderr:ssh: connect to host drop.ceph.com port 22: Connection timed out
2026-01-22T00:39:48.462 INFO:tasks.workunit.client.0.trial167.stderr:^MConnection closed.
2026-01-22T00:39:48.462 INFO:tasks.workunit.client.0.trial167.stderr:Connection closed
2026-01-22T00:39:48.466 DEBUG:teuthology.orchestra.run:got remote process result: 255
2026-01-22T00:39:48.467 INFO:tasks.workunit:Stopping ['post-file.sh'] on client.0...

I'm guessing this is related to the lab migration.

Actions #1

Updated by Laura Flores about 2 months ago

/a/yuriw-2026-01-23_21:16:01-rados-wip-yuri13-testing-2026-01-23-1852-reef-distro-default-trial/15541

Actions #2

Updated by David Galloway about 2 months ago

Yes, drop.ceph.com will need to be rebuilt in IBM Cloud. IBM Security is not going to allow incoming SSH connections to the lab under any circumstances especially using a shared private SSH key.

@adam kraitman can you work on standing up a replacement service for drop.ceph.com in IBM Cloud please? Probably just a VM with access to very cheap slow storage.

Actions #3

Updated by David Galloway about 2 months ago

  • Assignee set to adam kraitman
Actions #4

Updated by adam kraitman about 2 months ago

  • Status changed from New to In Progress

Yeah but it will be cheaper in aws since we already have a running cluster there, it won't cost us anything to add drop to that env

Actions #6

Updated by Sridhar Seshasayee about 2 months ago

/a/skanta-2026-01-27_05:35:03-rados-wip-bharath1-testing-2026-01-26-1242-distro-default-trial/
2 jobs: ['19747', '19776']

Actions #7

Updated by Laura Flores about 2 months ago

/a/skanta-2026-01-28_09:27:17-rados-wip-bharath2-testing-2026-01-27-0639-squid-distro-default-trial/23115

Actions #8

Updated by Nitzan Mordechai about 2 months ago

/a/yuriw-2026-01-29_18:33:05-rados-wip-yuri2-testing-2026-01-28-1643-tentacle-distro-default-trial/26517
/a/yuriw-2026-01-29_18:33:05-rados-wip-yuri2-testing-2026-01-28-1643-tentacle-distro-default-trial/26502

Actions #9

Updated by Aishwarya Mathuria about 1 month ago

/a/skanta-2026-01-30_23:46:16-rados-wip-bharath7-testing-2026-01-29-2016-distro-default-trial/28580
/a/skanta-2026-01-30_23:46:16-rados-wip-bharath7-testing-2026-01-29-2016-distro-default-trial/28559

Actions #10

Updated by adam kraitman about 1 month ago

@David Galloway Upload and download to drop.ceph.com will be done only using ssh ?

Actions #11

Updated by Connor Fawcett about 1 month ago

/a/skanta-2026-01-27_07:02:07-rados-wip-bharath3-testing-2026-01-26-1323-distro-default-trial/19849
/a/skanta-2026-01-27_07:02:07-rados-wip-bharath3-testing-2026-01-26-1323-distro-default-trial/19872

Actions #12

Updated by Kamoltat (Junior) Sirivadhna about 1 month ago

  • Subject changed from drop.ceph.com is unreachable to drop.ceph.com is unreachable (ceph-post-file)
Actions #13

Updated by Kamoltat (Junior) Sirivadhna about 1 month ago

/a/skanta-2026-01-29_13:10:13-rados-wip-bharath6-testing-2026-01-29-0855-squid-distro-default-trial/
[25749, 25764]

Actions #14

Updated by Nitzan Mordechai about 1 month ago

/a/yuriw-2026-02-04_23:08:40-rados-wip-yuri3-testing-2026-02-04-1948-tentacle-distro-default-trial/
['35467', '35329']

Actions #15

Updated by Aishwarya Mathuria about 1 month ago

/a/skanta-2026-02-07_00:02:26-rados-wip-bharath7-testing-2026-02-06-0906-distro-default-trial
['39110', '39127']

Actions #16

Updated by Lee Sanders about 1 month ago

/a/skanta-2026-01-29_02:19:11-rados-wip-bharath5-testing-2026-01-28-2018-distro-default-trial/24632
/a/skanta-2026-01-29_02:19:11-rados-wip-bharath5-testing-2026-01-28-2018-distro-default-trial/24770

Actions #17

Updated by Lee Sanders about 1 month ago

/a/skanta-2026-01-29_13:05:02-rados-wip-bharath5-testing-2026-01-28-2018-distro-default-trial/
['25703', '25727']

Actions #18

Updated by Jaya Prakash about 1 month ago

/a/jayaprakash-2026-02-06_12:54:34-rados-jaya-bs-testing-05-02-2026-distro-default-trial/38064

Actions #19

Updated by Nitzan Mordechai about 1 month ago

/a/yaarit-2026-02-08_02:25:21-rados-wip-rocky10-branch-of-the-day-2026-02-06-1770413686-distro-default-trial/39959

Actions #20

Updated by Nitzan Mordechai about 1 month ago

/a/nmordech-2026-02-09_08:34:53-rados-wip-rocky10-branch-of-the-day-2026-02-06-1770413686-distro-default-trial/41145
/a/nmordech-2026-02-09_08:34:53-rados-wip-rocky10-branch-of-the-day-2026-02-06-1770413686-distro-default-trial/41296

Actions #21

Updated by Nitzan Mordechai about 1 month ago

/a/yaarit-2026-02-10_02:34:33-rados-wip-rocky10-branch-of-the-day-2026-02-09-1770676549-distro-default-trial/[42194,42345 ]

Actions #22

Updated by Aishwarya Mathuria about 1 month ago

/a/skanta-2026-02-05_03:38:32-rados-wip-bharath2-testing-2026-02-03-0542-distro-default-trial
['35642', '35663']

Actions #23

Updated by Nitzan Mordechai about 1 month ago

/a/yaarit-2026-02-10_23:48:52-rados-wip-rocky10-branch-of-the-day-2026-02-09-1770676549-distro-default-trial/
['44481', '44307']

Actions #24

Updated by Laura Flores about 1 month ago

/a/yuriw-2026-01-27_16:01:09-rados-reef-release-distro-default-trial/21520

Actions #25

Updated by Aishwarya Mathuria about 1 month ago

/a/yuriw-2026-02-17_20:43:43-rados-wip-yuri6-testing-2026-02-17-1732-squid-distro-default-trial/
['53892', '53866']

Actions #26

Updated by Sridhar Seshasayee 26 days ago

/a/skanta-2026-02-22_05:18:48-rados-wip-bharath21-testing-2026-02-20-1039-distro-default-trial/
2 jobs: ['63344', '63356']

Actions #27

Updated by Lee Sanders 26 days ago

/a/skanta-2026-02-07_14:54:11-rados-wip-bharath5-testing-2026-02-06-2052-distro-default-trial/
['39470', '39488']

Actions #28

Updated by Jaya Prakash 25 days ago

2 jobs: ['62188', '62050']
/a/yuriw-2026-02-21_00:21:18-rados-wip-yuri3-testing-2026-02-19-1610-tentacle-distro-default-trial

Actions #29

Updated by Laura Flores 23 days ago

/a/nmordech-2026-02-25_11:36:23-rados-wip-rocky10-branch-of-the-day-2026-02-24-1771941190-distro-default-trial/70140

Actions #30

Updated by Sridhar Seshasayee 19 days ago

/a/sseshasa-2026-02-26_14:56:45-rados-wip-sseshasa-testing-2026-02-26-1772100687-distro-default-trial/
['72450', '72312']

Actions #31

Updated by Nitzan Mordechai 19 days ago

/a/yaarit-2026-02-26_20:20:34-rados-wip-rocky10-branch-of-the-day-2026-02-26-1772108951-distro-default-trial/
2 jobs: ['73032', '72881']

Actions #32

Updated by Nitzan Mordechai 18 days ago

/a/yaarit-2026-03-03_02:44:53-rados-wip-rocky10-branch-of-the-day-2026-03-02-1772457124-tentacle-distro-default-trial/
2 jobs: ['77778', '77627']

Actions #33

Updated by Nitzan Mordechai 17 days ago

/a/yaarit-2026-03-04_01:35:25-rados-wip-rocky10-branch-of-the-day-2026-03-03-1772543694-tentacle-distro-default-trial/
2 jobs: ['80385', '80536']

Actions #34

Updated by Laura Flores 16 days ago

/a/yaarit-2026-03-05_02:43:32-rados-wip-rocky10-branch-of-the-day-2026-03-04-1772633736-distro-default-trial/86244

Actions #35

Updated by Nitzan Mordechai 13 days ago

/a/yuriw-2026-03-07_15:37:55-rados-wip-rocky10-branch-of-the-day-2026-03-06-1772840606-distro-default-trial/
2 jobs: ['93069', '93223']

Actions #36

Updated by Aishwarya Mathuria 10 days ago

/a/skanta-2026-03-07_15:39:05-rados-wip-bharath4-testing-2026-03-05-1456-tentacle-distro-default-trial/93060
/a/skanta-2026-03-07_15:39:05-rados-wip-bharath4-testing-2026-03-05-1456-tentacle-distro-default-trial/93033

Actions #37

Updated by Connor Fawcett 9 days ago

/a/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68645
/a/yuriw-2026-02-24_16:22:43-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/68634

Actions #38

Updated by Sridhar Seshasayee 8 days ago

/a/skanta-2026-03-04_23:53:38-rados-wip-bharath1-testing-2026-03-04-1011-distro-default-trial
2 jobs: ['85622', '85641']

Actions #39

Updated by Jaya Prakash 5 days ago

/a/jayaprakash-2026-03-09_16:16:59-rados-wip-jaya-bs-testing-06-03-2025-distro-default-trial/95074

Actions #40

Updated by Lee Sanders 2 days ago

/a/yuriw-2026-02-26_01:51:16-rados-wip-yuri9-testing-2026-02-25-1600-squid-distro-default-trial/
['71054', '70918']

/a/yuriw-2026-02-26_16:03:22-rados-wip-yuri9-testing-2026-02-25-1600-squid-distro-default-trial/
['72539', '72554']

Actions

Also available in: Atom PDF