debian,ceph.spec: split ceph-osd into shared base and implementation packages#65782
debian,ceph.spec: split ceph-osd into shared base and implementation packages#65782
Conversation
|
win32 build failed: i don't think it's relevant though. |
|
jenkins test windows |
|
jenkins test make check arm64 |
1 similar comment
|
jenkins test make check arm64 |
There was a problem hiding this comment.
I strongly agree that having both flavors "hidden" behind ceph-osd is confusing from user prescriptive. This version is much more transparent and reliable.
Another strong advantage here is that we could always enable crimson as part of ceph images - without needing two separate images/packages for each build (!).
I think it might be worth mentioning that bare-metal deployments still won't be able to coexist (as the binaries still conflict), right?
As far as concerns go:
-
One blocker here would be - #65209 as the alienized version of Bluestore used by Crimson is not compatible with classic-osd - so we should make sure we disallow wrong flavor used by an existing store.
-
This is opens a gate for user mistakes switching the two binaries using manual packaging operations. I think we should disallow osds to boot in case the binary was replaced from classic to crimson or vice versa (leaving upgrades aside). Perhaps the PR mentioned above would be a good solution to this issue. @tchaikov, what do you think?
CC: @rzarzynski, @athanatos, @jdurgin
| %{_bindir}/crimson-objectstore-tool | ||
|
|
||
| %post crimson-osd | ||
| %{_sbindir}/update-alternatives --install %{_bindir}/ceph-osd ceph-osd \ |
There was a problem hiding this comment.
We are installing ceph-crimson-osd with lower priority (50) then classic (100), which means to use crimson we would have to manually update-alternatives --set?
being tested at https://shaman.ceph.com/builds/ceph/wip-build-crimson-osd-kefu-10-07/e4b6588dfa11d429e02bc777d1b2c5b5dcd325d0/
Test suite would also need to set the relevant binary for crimson iiuc
There was a problem hiding this comment.
correct. i haven't updated the tests for this change. i should have mark this change as a draft. but i wanted to have some inputs on the general strategy of the packaging.
There was a problem hiding this comment.
I think we should prioritize this change -- I can help with testing side if needed!
There was a problem hiding this comment.
@Matan-B thanks for offering the help. i just updated the teuthology test to use the new package name. let's see if there is any objection, once it's good. we should certainly test it ASAP.
debian/control
Outdated
| ceph-test (<< 12.2.2-14), | ||
| ceph-osd (<< 17.0.0) | ||
| ceph-osd (<< 20.1.1) | ||
| Conflicts: ceph-crimson-osd |
There was a problem hiding this comment.
Is it thinkable to have both ceph-crimson-osd and ceph-classic-osd the same time in a single system?
Moving the decision post install-time would make Containerization People's life much easier.
What confuses me is:
The two implementation packages conflict with each other but both depend on ceph-osd for shared resources.
Is a package manager going to allow those 2 conflicted pkgs be installed the same time?
There was a problem hiding this comment.
Is it thinkable to have both ceph-crimson-osd and ceph-classic-osd the same time in a single system?
i understand that this arrangement would be more flexible. but that'd require us to prepare two set of service units and sysctl configurations. and even worse -- ceph-classic-osd would be competing with ceph-crimson-osd for CPU cycles and memory. please note, seastar preallocates memory beforehand when using its builtin allocator. this would be a challenge for administrators to planning and to debugging performance issues.
There was a problem hiding this comment.
What confuses me is:
The two implementation packages conflict with each other but both depend on ceph-osd for shared resources.
Is a package manager going to allow those 2 conflicted pkgs be installed the same time?
no. it does not. the idea is that we don't allow classic osd and crimson osd to colocate on the same node. but since they share the same set of utilities and service unit, we let both of them depend on the package providing these tools. the sharing is not an arrangement to "share" on resource the same node, but to share the package dependency.
There was a problem hiding this comment.
Perhaps we could rename the title
Unify Crimson and Classic into a single package
Moreover, I think once we implement the testing side with actual usage of update-alternatives --set it would be easier to understand the change.
There was a problem hiding this comment.
after a second thought, i am dropping the "Conflicts" between ceph-classic-osd and ceph-crimson-osd to facilitate the containerization of crimson + classic
There was a problem hiding this comment.
Unify Crimson and Classic into a single package
strictly speaking, we are not unifying them into a single package. instead, ceph-osd is now split into ceph-osd and ceph-classic-osd. crimson is now a separate package. if we really want to use "unify" in the title, we are unifying two flavors. but it would be more confusing, as the idea of "flavor" is not quite related to the change here.
That's intentional. As I am afraid that collocating these two different types of OSDs on the same bare-metal host could lead to confusion and issues that are difficult to troubleshoot. Keeping them separate helps maintain clearer system boundaries.
i agree it's important to prevent user from switching from classic osd to crimson on a certain node or the other way around. but why do you think it's a blocker of this change? that's a general design consideration for crimson -- a guard rail for misconfiguration.
i see. making classic-osd and crimson "interchangeable" does make it simpler. but, please note, before this change, user is already allowed to install crimson osd to node where classic osd was installed. actually this is how packaging works -- to allow user to install packages. in short, i think these two problems are orthogonal. we should address this problem separately, but not with packaging. #65209 could be a good solution, but i haven't got a chance to look into it yet.
|
Matan-B
left a comment
There was a problem hiding this comment.
in short, i think these two problems are orthogonal. we should address this problem separately, but not with packaging. #65209 could be a good solution, but i haven't got a chance to look into it yet.
Agree it shouldn't be a blocker, I think we should move forward with this.
Next steps for this PR could be:
- Update Crimson's suite to use the new package
- Update Crimson Docs (how to use the crimson binary)
Possibly also: - Enable (WITH_CRIMSON) Crimson by default
- Stop building Crimson flavor building in CI/Shaman
@tchaikov, @rzarzynski - WDYT?
Please let me know if I can help with any of the above if agreed.
|
changelog
|
|
@Matan-B Hi Matan, thanks for the feedback. I agree with your overall direction. Here's my plan for addressing your suggestions:
I'm updating the teuthology suites in the latest revision of this PR to implement this change.
I'd prefer to handle this as a follow-up PR that also renames the executables:
This naming change might be controversial, so I think it deserves separate discussion.
We currently use an opt-in policy for building Crimson because not all distros have C++ compilers capable of building Seastar with coroutines. Changing this could break downstream builds, so I'd prefer to defer this to a follow-up change.
I don't think we can implement this change in this repo. I'll coordinate with the team on an upcoming change in ceph-build.
|
e4b6588 to
1d62f09
Compare
1d62f09 to
0003153
Compare
There was a problem hiding this comment.
LGTM! Thank you for replying to the concerns brought up in previous comments.
Summarizing future steps here:
-
A follow-up PR that will suggest to also rename the executables:
crimson-osd → ceph-crimson-osd
ceph-osd → ceph-classic-osd
Once this PR is approved we should be able to also update the documentation with how to enable/use the Crimson osd binary with the new packaging. -
Enabling WITH_CRIMSON (if possible, by default) in order to stop building dedicated Crimson builds in CI could be discussed in a follow up PR (ceph-build repo).
nit: PR description mentions "conflicts" which is not longer true iiuc.
0003153 to
922a2fa
Compare
|
@Matan-B seems we need more changes to ready this transition. i am looking at https://shaman.ceph.com/builds/ceph/wip-build-crimson-osd-kefu-02/b72a63cd24eb87a1df711ca50783433e975d50ad/default/455473/ . where we enabled |
Matan-B
left a comment
There was a problem hiding this comment.
@Matan-B seems we need more changes to ready this transition. i am looking at https://shaman.ceph.com/builds/ceph/wip-build-crimson-osd-kefu-02/b72a63cd24eb87a1df711ca50783433e975d50ad/default/455473/ . where we enabled
pkg.ceph.crimsonbuild profile on ubuntu jammy because the branch name contains "crimson" even the flavor is not any one of "crimson-debug" , "crimson-release" and "crimson".
++ with_crimson=true
++ with_crimson=true
++ local with_crimson=true
++ with_crimson=true
....
++ ci_debug 'Running munge_debian_control() in install-deps.sh'
++ in_jenkins
++ '[' -n /ceph ']'
++ echo 'CI_DEBUG: Running munge_debian_control() in install-deps.sh'
CI_DEBUG: Running munge_debian_control() in install-deps.sh
++ backports=
+++ munge_debian_control '24.04.3 LTS (Noble Numbat)' debian/control
+++ local 'version=24.04.3 LTS (Noble Numbat)'
+++ shift
+++ local control=debian/control
+++ case "$version" in
+++ echo debian/control
++ control=debian/control
++ case "$VERSION" in
++ build_profiles=
++ false
++ true
++ build_profiles+=,pkg.ceph.crimson
++ false
I'm assuming the ARG WITH_CRIMSON=false will do, but will this impact pr makecheks?
Dockerfile.build
Outdated
| ARG SCCACHE_VERSION=v0.8.2 | ||
| ARG SCCACHE_REPO=https://github.com/mozilla/sccache | ||
| ARG WITH_CRIMSON=true | ||
| ARG WITH_CRIMSON=false |
There was a problem hiding this comment.
Will this disable Crimson makecheck builds?
There was a problem hiding this comment.
no, it does not. i just checked the log file of the "make check" run of this PR:
$ grep -E '(crimson|seastar)' \#168804.txt | grep ': unittest-'
Start 266: unittest-crimson-backfill
Start 267: unittest-seastar-buffer
Start 268: unittest-seastar-denc
Start 269: unittest-seastar-socket
Start 270: unittest-seastar-messenger
Start 271: unittest-seastar-alienstore-thread-pool
Start 272: unittest-seastar-config
167/320 Test #268: unittest-seastar-denc ..................... Passed 0.37 sec
Start 273: unittest-seastar-perfcounters
Start 274: unittest-seastar-lru
Start 275: unittest-seastar-calc-subsets
Start 278: unittest-seastar-messenger-thrash
173/320 Test #274: unittest-seastar-lru ...................... Passed 0.22 sec
Start 279: unittest-seastar-errorator
Start 280: unittest-crimson-coroutine
178/320 Test #275: unittest-seastar-calc-subsets ............. Passed 0.55 sec
182/320 Test #267: unittest-seastar-buffer ................... Passed 1.18 sec
193/320 Test #269: unittest-seastar-socket ................... Passed 5.81 sec
197/320 Test #272: unittest-seastar-config ................... Passed 6.64 sec
199/320 Test #271: unittest-seastar-alienstore-thread-pool ... Passed 7.27 sec
203/320 Test #273: unittest-seastar-perfcounters ............. Passed 8.32 sec
204/320 Test #266: unittest-crimson-backfill ................. Passed 9.64 sec
208/320 Test #279: unittest-seastar-errorator ................ Passed 11.14 sec
209/320 Test #280: unittest-crimson-coroutine ................ Passed 11.34 sec
235/320 Test #270: unittest-seastar-messenger ................ Passed 30.56 sec
266/320 Test #278: unittest-seastar-messenger-thrash ......... Passed 101.29 secsee also https://jenkins.ceph.com/job/ceph-pull-requests/168804/
There was a problem hiding this comment.
Line 420 in a9b8933
I don't think this does what you think it does. WITH_CRIMSON=false enabled crimson.
I wrote a long response that github ate. In short:
The WITH_CRIMSON=true default was causing all builds to attempt building
crimson, including distributions that don't meet the GCC 13+ requirement
(e.g., Ubuntu Jammy with GCC 11.4.0).
The Jenkins CI pipeline correctly excludes crimson flavors from non-centos9
distributions via matrix excludes, and explicitly sets WITH_CRIMSON=true
via .env only for crimson-* flavors. However, the Dockerfile ARG default
was overriding this logic for non-crimson builds where WITH_CRIMSON was
not explicitly set.
By changing the default to false, crimson builds now only occur when
explicitly requested via the WITH_CRIMSON environment variable, which
aligns with the CI pipeline's intended behavior.
I think your info is out of date. The containerfile defaults are for make check. Everything else (CI jobs, packages) can use the build image variant feature of BWC which is specifically designed to use a slimmer subset of packages, but the defaults are for make check (the primary use case for BWC).
If you change this we're likely going to break casual users of BWC who don't want to fiddle with a bunch of CLI options just to run make check in a container (including myself).
|
In the matter of package naming: the flavor name in the middle (the infix notation) seems a bit unfriendly towards sorting and tab-completion. How about:
|
301c179 to
9299468
Compare
@tchaikov the issue is also coming up in the nightly runs, where all tests are running on the same commit. The affected tests are |
@ljflores, I've pushed a fix #66268 fixing |
With ceph#65782 merged, Crimson could be now built as part of the default RPM build flavor. The binaries ceph-osd-crimson and ceph-osd-classic are managed via `update-alternatives` and selected based on priority: - ceph-osd-classic keeps the **highest** priority (100) and remains the default. - Switching to crimson is possible via: update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson Existing classic behavior remains unchanged unless the above command is explicitly executed. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, Crimson could be now built as part of the default RPM build flavor. The binaries ceph-osd-crimson and ceph-osd-classic are managed via `update-alternatives` and selected based on priority: - ceph-osd-classic keeps the **highest** priority (100) and remains the default. - Switching to crimson is possible via: update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson Existing classic behavior remains unchanged unless the above command is explicitly executed. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, Crimson could be now built as part of the default RPM build flavor. The binaries ceph-osd-crimson and ceph-osd-classic are managed via `update-alternatives` and selected based on priority: - ceph-osd-classic keeps the **highest** priority (100) and remains the default. - Switching to crimson is possible via: update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson Existing classic behavior remains unchanged unless the above command is explicitly executed. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, Crimson could be now built as part of the default RPM build flavor. The binaries ceph-osd-crimson and ceph-osd-classic are managed via `update-alternatives` and selected based on priority: - ceph-osd-classic keeps the **highest** priority (100) and remains the default. - Switching to crimson is possible via: update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson Existing classic behavior remains unchanged unless the above command is explicitly executed. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, Crimson could be now built as part of the default RPM build flavor. The binaries ceph-osd-crimson and ceph-osd-classic are managed via `update-alternatives` and selected based on priority: - ceph-osd-classic keeps the **highest** priority (100) and remains the default. - Switching to crimson is possible via: update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson Existing classic behavior remains unchanged unless the above command is explicitly executed. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Previously we were using a crimson dedicated images, `ceph-osd` is no longer overriden when Crimson is enabled. See: ceph#65782 for packaged based deployments we are now using: ``` sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson ``` containerized deployments should also be able to use crimson-osd when both osd are available in the image. extra_container_args would allow to pass additional arguments directly to the container at runtime and applied (ceph orch apply) Fixes: https://tracker.ceph.com/issues/73970 Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Previously we were using a crimson dedicated images, `ceph-osd` is no longer overriden when Crimson is enabled. See: ceph#65782 for packaged based deployments we are now using: ``` sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson ``` containerized deployments should also be able to use crimson-osd when both osd are available in the image. extra_container_args would allow to pass additional arguments directly to the container at runtime and applied (ceph orch apply) Fixes: https://tracker.ceph.com/issues/73970 Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Previously we were using a crimson dedicated images, `ceph-osd` is no longer overriden when Crimson is enabled. See: ceph#65782 for packaged based deployments we are now using: ``` sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson ``` containerized deployments should also be able to use crimson-osd when both osd are available in the image. extra_container_args would allow to pass additional arguments directly to the container at runtime and applied (ceph orch apply) Fixes: https://tracker.ceph.com/issues/73970 Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Previously we were using a crimson dedicated images, `ceph-osd` is no longer overriden when Crimson is enabled. See: ceph#65782 for packaged based deployments we are now using: ``` sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson ``` containerized deployments should also be able to use crimson-osd when both osd are available in the image. extra_container_args would allow to pass additional arguments directly to the container at runtime and applied (ceph orch apply) Fixes: https://tracker.ceph.com/issues/73970 Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Previously we were using a crimson dedicated images, `ceph-osd` is no longer overriden when Crimson is enabled. See: ceph#65782 for packaged based deployments we are now using: ``` sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson ``` containerized deployments should also be able to use crimson-osd when both osd are available in the image. extra_container_args would allow to pass additional arguments directly to the container at runtime and applied (ceph orch apply) Fixes: https://tracker.ceph.com/issues/73970 Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Sorry for commenting on a merged PR. I'm seeing related failures in fs suite. https://tracker.ceph.com/issues/74026. Trying a fix here: #66475 but still needs work. |
Replied in the PR, thanks! |
With ceph#65782 merged, Crimson could be now built as part of the default RPM build flavor. The binaries ceph-osd-crimson and ceph-osd-classic are managed via `update-alternatives` and selected based on priority: - ceph-osd-classic keeps the **highest** priority (100) and remains the default. - Switching to crimson is possible via: update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson Existing classic behavior remains unchanged unless the above command is explicitly executed. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
|
@tchaikov @Matan-B i'm having trouble getting the rgw/upgrade suite to pass after this change. i'm guessing that all of the other upgrade jobs are using cephadm, but the rgw suite is still package-based (edit: i see that @vshankar is seeing the same in #66475) in #66341, i'm trying to work around errors like the root cause appears to be these errors when the upgrade installs ceph-osd-classic and ceph-osd-crimson:
the original ceph-osd binary gets removed later in the upgrade, leading to failures when trying to restart the osds:
|
Previously we were using a crimson dedicated images, `ceph-osd` is no longer overriden when Crimson is enabled. See: ceph/ceph#65782 for packaged based deployments we are now using: ``` sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson ``` containerized deployments should also be able to use crimson-osd when both osd are available in the image. extra_container_args would allow to pass additional arguments directly to the container at runtime and applied (ceph orch apply) Fixes: https://tracker.ceph.com/issues/73970 Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, upgrading ceph-osd would need to replace the previous ceph-osd existing on the machine. Otherwise, we won't be able to symlink the newly installed package: ``` 2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.479 INFO:teuthology.orchestra.run.smithi077.stdout:failed to link /usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd exists and it is not a symlink ``` Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, upgrading ceph-osd would need to replace the previous ceph-osd existing on the machine. Otherwise, we won't be able to symlink the newly installed package: ``` 2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.479 INFO:teuthology.orchestra.run.smithi077.stdout:failed to link /usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd exists and it is not a symlink ``` Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, upgrading ceph-osd would need to replace the previous ceph-osd existing on the machine. Otherwise, we won't be able to symlink the newly installed package: ``` 2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.479 INFO:teuthology.orchestra.run.smithi077.stdout:failed to link /usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd exists and it is not a symlink ``` Note: debian/control ceph-osd-classic already had Replace and Breaks: - Breaks is replaced with Conflicts to not allow coexistence. - Release version is bumped up to be relevant for latest main Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph/ceph#65782 merged, upgrading ceph-osd would need to replace the previous ceph-osd existing on the machine. Otherwise, we won't be able to symlink the newly installed package: ``` 2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.479 INFO:teuthology.orchestra.run.smithi077.stdout:failed to link /usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd exists and it is not a symlink ``` Note: debian/control ceph-osd-classic already had Replace and Breaks: - Breaks is replaced with Conflicts to not allow coexistence. - Release version is bumped up to be relevant for latest main Signed-off-by: Matan Breizman <mbreizma@redhat.com> (cherry picked from commit 16cc623)
With ceph/ceph#65782 merged, upgrading ceph-osd would need to replace the previous ceph-osd existing on the machine. Otherwise, we won't be able to symlink the newly installed package: ``` 2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.479 INFO:teuthology.orchestra.run.smithi077.stdout:failed to link /usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd exists and it is not a symlink ``` Note: debian/control ceph-osd-classic already had Replace and Breaks: - Breaks is replaced with Conflicts to not allow coexistence. - Release version is bumped up to be relevant for latest main Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph/ceph#65782 merged, upgrading ceph-osd would need to replace the previous ceph-osd existing on the machine. Otherwise, we won't be able to symlink the newly installed package: ``` 2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.479 INFO:teuthology.orchestra.run.smithi077.stdout:failed to link /usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd exists and it is not a symlink ``` Note: debian/control ceph-osd-classic already had Replace and Breaks: - Breaks is replaced with Conflicts to not allow coexistence. - Release version is bumped up to be relevant for latest main Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, upgrading ceph-osd would need to replace the previous ceph-osd existing on the machine. Otherwise, we won't be able to symlink the newly installed package: ``` 2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.479 INFO:teuthology.orchestra.run.smithi077.stdout:failed to link /usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd exists and it is not a symlink ``` Note: debian/control ceph-osd-classic already had Replace and Breaks: - Breaks is replaced with Conflicts to not allow coexistence. - Release version is bumped up to be relevant for latest main Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, Crimson could be now built as part of the default RPM build flavor. The binaries ceph-osd-crimson and ceph-osd-classic are managed via `update-alternatives` and selected based on priority: - ceph-osd-classic keeps the **highest** priority (100) and remains the default. - Switching to crimson is possible via: update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson Existing classic behavior remains unchanged unless the above command is explicitly executed. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, upgrading ceph-osd would need to replace the previous ceph-osd existing on the machine. Otherwise, we won't be able to symlink the newly installed package: ``` 2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.479 INFO:teuthology.orchestra.run.smithi077.stdout:failed to link /usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd exists and it is not a symlink ``` Note: debian/control ceph-osd-classic already had Replace and Breaks: - Breaks is replaced with Conflicts to not allow coexistence. - Release version is bumped up to be relevant for latest main Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, Crimson could be now built as part of the default RPM build flavor. The binaries ceph-osd-crimson and ceph-osd-classic are managed via `update-alternatives` and selected based on priority: - ceph-osd-classic keeps the **highest** priority (100) and remains the default. - Switching to crimson is possible via: update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson Existing classic behavior remains unchanged unless the above command is explicitly executed. Signed-off-by: Matan Breizman <mbreizma@redhat.com>
With ceph#65782 merged, upgrading ceph-osd would need to replace the previous ceph-osd existing on the machine. Otherwise, we won't be able to symlink the newly installed package: ``` 2025-12-05T21:09:20.472 INFO:teuthology.orchestra.run.smithi077.stdout: Installing : ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.478 INFO:teuthology.orchestra.run.smithi077.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-4434.g8611241d.el9.x86_6 24/87 2025-12-05T21:09:20.479 INFO:teuthology.orchestra.run.smithi077.stdout:failed to link /usr/bin/ceph-osd -> /etc/alternatives/ceph-osd: /usr/bin/ceph-osd exists and it is not a symlink ``` Note: debian/control ceph-osd-classic already had Replace and Breaks: - Breaks is replaced with Conflicts to not allow coexistence. - Release version is bumped up to be relevant for latest main Signed-off-by: Matan Breizman <mbreizma@redhat.com>
Previously, ceph-osd packaging had two mutually exclusive flavors that could only be built one at a time: one with classic OSD and another with crimson OSD. Both provided /usr/bin/ceph-osd, making them impossible to coexist and confusing from a user perspective. This commit restructures the packaging to enable both implementations to coexist on the same system:
The two implementation packages conflict with each other but both depend on ceph-osd for shared resources.
Changes:
Debian packaging:
RPM packaging:
Upgrade behavior:
Users upgrading from older versions will automatically get ceph-classic-osd due to the Recommends directive, maintaining backward compatibility. Users can explicitly choose crimson by installing ceph-osd-crimson, which will conflict out classic. Switching between implementations is supported via standard package operations, with the alternatives system ensuring /usr/bin/ceph-osd always points to the active implementation.
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an
xbetween the brackets:[x]. Spaces and capitalization matter when checking off items this way.Checklist
Show available Jenkins commands
jenkins test classic perfJenkins Job | Jenkins Job Definitionjenkins test crimson perfJenkins Job | Jenkins Job Definitionjenkins test signedJenkins Job | Jenkins Job Definitionjenkins test make checkJenkins Job | Jenkins Job Definitionjenkins test make check arm64Jenkins Job | Jenkins Job Definitionjenkins test submodulesJenkins Job | Jenkins Job Definitionjenkins test dashboardJenkins Job | Jenkins Job Definitionjenkins test dashboard cephadmJenkins Job | Jenkins Job Definitionjenkins test apiJenkins Job | Jenkins Job Definitionjenkins test docsReadTheDocs | Github Workflow Definitionjenkins test ceph-volume allJenkins Jobs | Jenkins Jobs Definitionjenkins test windowsJenkins Job | Jenkins Job Definitionjenkins test rook e2eJenkins Job | Jenkins Job DefinitionYou must only issue one Jenkins command per-comment. Jenkins does not understand
comments with more than one command.