Skip to content

qa/suites/rados/encoder: exclude ceph-osd-classic when installing LTS…#66372

Merged
tchaikov merged 1 commit intoceph:mainfrom
tchaikov:wip-qa-encoder-exclude
Nov 24, 2025
Merged

qa/suites/rados/encoder: exclude ceph-osd-classic when installing LTS…#66372
tchaikov merged 1 commit intoceph:mainfrom
tchaikov:wip-qa-encoder-exclude

Conversation

@tchaikov
Copy link
Contributor

@tchaikov tchaikov commented Nov 22, 2025 •
edited
Loading

… releases

In a37b5b5, the ceph-osd-classic package was added to qa/packages/packages.yaml. The "install" task uses this file as the default package list for all branches, including LTS releases like Reef.

However, a37b5b5 only exists in the main branch and won't be backported to LTS branches. This causes installation failures in the rados/encoder test suite, which verifies forward compatibility by installing LTS releases and testing whether they can decode the latest corpus.

Exclude ceph-osd-classic from LTS installations to ensure the test suite can successfully install ceph-dencoder, which is required for the interoperability tests.

Fixes: https://tracker.ceph.com/issues/73957

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands

You must only issue one Jenkins command per-comment. Jenkins does not understand
comments with more than one command.

@tchaikov tchaikov requested review from Matan-B and ljflores November 22, 2025 00:57
@tchaikov tchaikov requested a review from a team as a code owner November 22, 2025 00:57
@github-actions github-actions bot added the core label Nov 22, 2025
@tchaikov
Copy link
Contributor Author

tchaikov commented Nov 22, 2025

as a follow-up change, i will create another PR to customize the packages.yaml for the qa/suites/rados/encoder test suite, so it just installs ceph-common which includes the only executable ceph-dencoder used by this test.

@tchaikov
Copy link
Contributor Author

tchaikov commented Nov 22, 2025 •
edited
Loading

tested at https://pulpito.ceph.com/kchai-2025-11-22_01:20:00-rados:encoder-main-distro-default-smithi/

$ teuthology-suite  --force-priority --subset 1033/99999 --machine-type smithi --suite rados:encoder  --sha1 fb68c6868422015553f389c941aebd063a5f1b4c --email tchaikov@gmail.com --seed 2477 --limit 1 --suite-repo https://github.com/tchaikov/ceph.git --suite-branch wip-qa-encoder-exclude

failed:


2025-11-22T01:37:43.533 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull
2025-11-22T01:37:44.421 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy...
2025-11-22T01:37:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:Non-zero exit code 125 from /bin/podman pull quay.ceph.io/ceph-ci/ceph:quincy
2025-11-22T01:37:44.610 INFO:teuthology.orchestra.run.smithi078.stderr:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy...
2025-11-22T01:37:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:/bin/podman: stderr Error: unable to copy from source docker://quay.ceph.io/ceph-ci/ceph:quincy: initializing source docker://quay.ceph.io/ceph-ci/ceph:quincy: reading manifest quincy in quay.ceph.io/ceph-ci/ceph: manifest unknown
2025-11-22T01:37:44.611 INFO:teuthology.orchestra.run.smithi078.stderr:ERROR: Failed command: /bin/podman pull quay.ceph.io/ceph-ci/ceph:quincy
2025-11-22T01:37:44.629 DEBUG:teuthology.orchestra.run:got remote process result: 1
2025-11-22T01:37:44.630 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_258eb6279f4d7fcd4b45c82e521f2a2e799d7f33/teuthology/contextutil.py", line 30, in nested
    vars.append(enter())
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_tchaikov_ceph_e27411b58359a2cfe3e3920b31b86dfd22f0f00d/qa/tasks/cephadm.py", line 553, in pull_image
    run.wait(ctx.cluster.run(args=cmd, wait=False))
  File "/home/teuthworker/src/git.ceph.com_teuthology_258eb6279f4d7fcd4b45c82e521f2a2e799d7f33/teuthology/orchestra/run.py", line 485, in wait
    proc.wait()
  File "/home/teuthworker/src/git.ceph.com_teuthology_258eb6279f4d7fcd4b45c82e521f2a2e799d7f33/teuthology/orchestra/run.py", line 161, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_teuthology_258eb6279f4d7fcd4b45c82e521f2a2e799d7f33/teuthology/orchestra/run.py", line 181, in _raise_for_status
    raise CommandFailedError(
teuthology.exceptions.CommandFailedError: Command failed on smithi078 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull'

but the install task completed, see the logging message from http://qa-proxy.ceph.com/teuthology/kchai-2025-11-22_02:18:28-rados:encoder-main-distro-default-smithi/8619492/teuthology.log

2025-11-22T02:36:28.691 INFO:teuthology.run_tasks:Running task print...
2025-11-22T02:36:28.701 INFO:teuthology.task.print:**** done install task...

the test failed when pulling quincy image from quay.ceph.io/ceph-ci/ceph:quincy. we failed to install quincy in some other integration tests as well, see http://qa-proxy.ceph.com/teuthology/yuriw-2025-11-19_15:50:15-upgrade:quincy-x-reef-release-distro-default-smithi/8612969/teuthology.log:

2025-11-19T16:28:36.637 INFO:teuthology.orchestra.run.smithi126.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy...
2025-11-19T16:28:36.928 INFO:teuthology.orchestra.run.smithi126.stderr:Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:quincy
2025-11-19T16:28:36.928 INFO:teuthology.orchestra.run.smithi126.stderr:/usr/bin/docker: stderr Error response from daemon: unknown: Tag quincy was deleted or has expired. To pull, revive via time machine
2025-11-19T16:28:36.929 INFO:teuthology.orchestra.run.smithi126.stderr:ERROR: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:quincy
2025-11-19T16:28:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:Non-zero exit code 1 from /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:quincy
2025-11-19T16:28:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:/usr/bin/docker: stderr Error response from daemon: unknown: Tag quincy was deleted or has expired. To pull, revive via time machine
2025-11-19T16:28:36.929 INFO:teuthology.orchestra.run.smithi005.stderr:ERROR: Failed command: /usr/bin/docker pull quay.ceph.io/ceph-ci/ceph:quincy

@yuriw hi Yuri, do you happen to know what happened to the quincy image hosted by quay.ceph.io ?

Matan-B
Matan-B reviewed Nov 23, 2025
View reviewed changes
Copy link
Contributor

@Matan-B Matan-B left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With #66229 merged, should we also exclude ceph-osd-crimson?

@tchaikov
qa/suites/rados/encoder: exclude ceph-osd-* when installing LTS releases
3fcc9e3 
In a37b5b5, the ceph-osd-classic and ceph-osd-crimson packages were
added to qa/packages/packages.yaml. The "install" task uses this file as
the default package list for all branches, including LTS releases like
Reef.

However, a37b5b5 only exists in the main branch and won't be backported
to LTS branches. This causes installation failures in the rados/encoder
test suite, which verifies forward compatibility by installing LTS
releases and testing whether they can decode the latest corpus.

Exclude ceph-osd-classic and ceph-osd-crimson from LTS installations to
ensure the test suite can successfully install ceph-dencoder, which is
required for the interoperability tests.

Fixes: https://tracker.ceph.com/issues/73957
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
@tchaikov tchaikov force-pushed the wip-qa-encoder-exclude branch from e27411b to 3fcc9e3 Compare November 24, 2025 06:48
@tchaikov
Copy link
Contributor Author

tchaikov commented Nov 24, 2025

With #66229 merged, should we also exclude ceph-osd-crimson?

thank you Matan! indeed. excluded as well.

@tchaikov
Copy link
Contributor Author

tchaikov commented Nov 24, 2025

changelog

  • exclude ceph-osd-crimson as well.

@tchaikov tchaikov requested a review from Matan-B November 24, 2025 06:50
@tchaikov
Copy link
Contributor Author

tchaikov commented Nov 24, 2025

tested at https://pulpito.ceph.com/kchai-2025-11-24_06:49:46-rados:encoder-main-distro-default-smithi/ . the test failed due to the same reason. see the analysis at #66372 (comment)

@tchaikov tchaikov merged commit 812196b into ceph:main Nov 24, 2025
13 checks passed
@tchaikov
Copy link
Contributor Author

tchaikov commented Nov 24, 2025

because this addresses a regression. i am merging it without waiting for the quorum of reviewers.

@tchaikov tchaikov deleted the wip-qa-encoder-exclude branch November 24, 2025 08:27
@github-actions
Copy link

github-actions bot commented Nov 24, 2025

This is an automated message by src/script/redmine-upkeep.py.

I have resolved the following tracker ticket due to the merge of this PR:

No backports are pending for the ticket. If this is incorrect, please update the tracker
ticket and reset to Pending Backport state.

Update Log: https://github.com/ceph/ceph/actions/runs/19627878369

@Matan-B Matan-B mentioned this pull request Dec 3, 2025
Merged
14 tasks
@NitzanMordhai NitzanMordhai mentioned this pull request Feb 3, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Matan-B Matan-B Matan-B approved these changes

@ljflores ljflores Awaiting requested review from ljflores

Assignees

No one assigned

Labels

core needs-qa tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@tchaikov @Matan-B