crimson(cephadm): port cephadm test suite to crimson-rados by mohit84 · Pull Request #66217 · ceph/ceph

mohit84 · 2025-11-12T11:23:13Z

In the classic OSD test suite both ubuntu and centos builds are supported. However the crimson OSD only provides centos based builds so the same test layout cannot be reused directly.

To enable cephadm testing with crimson, this change ports the relevant test suite to the crimson-rados directory. Instead of linking entire subdirectories (like .qa/suites/orch/cephadm/{osds,smoke,workunits,smoke-singlehost}), the crimson suite uses selective symlinks for the required yaml files and exclude ubuntu specific yaml files.

Fixes: https://tracker.ceph.com/issues/71946

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins test classic perf Jenkins Job | Jenkins Job Definition
jenkins test crimson perf Jenkins Job | Jenkins Job Definition
jenkins test signed Jenkins Job | Jenkins Job Definition
jenkins test make check Jenkins Job | Jenkins Job Definition
jenkins test make check arm64 Jenkins Job | Jenkins Job Definition
jenkins test submodules Jenkins Job | Jenkins Job Definition
jenkins test dashboard Jenkins Job | Jenkins Job Definition
jenkins test dashboard cephadm Jenkins Job | Jenkins Job Definition
jenkins test api Jenkins Job | Jenkins Job Definition
jenkins test docs ReadTheDocs | Github Workflow Definition
jenkins test ceph-volume all Jenkins Jobs | Jenkins Jobs Definition
jenkins test windows Jenkins Job | Jenkins Job Definition
jenkins test rook e2e Jenkins Job | Jenkins Job Definition

You must only issue one Jenkins command per-comment. Jenkins does not understand
comments with more than one command.

Matan-B

Suite structure looks good! Can we avoid the cephadm.py change?

Matan-B · 2025-11-12T11:29:20Z

qa/tasks/cephadm.py

+        flavor = config.get('flavor', 'default')
+        if flavor in ("crimson-debug", "crimson-release"):
+            log.info('Set crimson_cpu_num ....')
+            _shell(ctx, cluster_name, bootstrap_remote, [
+                   'ceph', 'config', 'set', 'osd', 'crimson_cpu_num', '4'])
+            log.info('Set enable_experimental feature to crimson ...')
+            _shell(ctx, cluster_name, bootstrap_remote, [
+                   'ceph', 'config', 'set', 'global',
+                   'enable_experimental_unrecoverable_data_corrupting_features', 'crimson'])
+            log.info('Set set-allow-crimson --yes-i-really-mean-it ...')
+            _shell(ctx, cluster_name, bootstrap_remote, [
+                   'ceph', 'osd', 'set-allow-crimson', '--yes-i-really-mean-it'])
+


These should already be set by crimson_qa_overrides:

conf: global: enable experimental unrecoverable data corrupting features: crimson mon: osd pool default crimson: true

Can we cleanup this bit?

Matan-B · 2025-11-12T11:36:08Z

Note: when testing this PR please --filter cephadm since the other directories shouldn't be impacted

Matan-B · 2025-11-12T11:40:25Z

qa/tasks/cephadm.py

+        if flavor in ("crimson-debug", "crimson-release"):
+            log.info('Set crimson_cpu_num ....')
+            _shell(ctx, cluster_name, bootstrap_remote, [
+                   'ceph', 'config', 'set', 'osd', 'crimson_cpu_num', '4'])


These should be set in a separate yaml to not edit common cephadm.yaml as much as possible
See how crimson-fixed-1.yaml sets crimson_cpu_num
We can introduce a new yaml in crimson-rados/cephadm dir that sets this.

Matan-B · 2025-11-12T11:43:14Z

qa/tasks/cephadm.py

+                   'ceph', 'config', 'set', 'osd', 'crimson_cpu_num', '4'])
+            log.info('Set set-allow-crimson --yes-i-really-mean-it ...')
+            _shell(ctx, cluster_name, bootstrap_remote, [
+                   'ceph', 'osd', 'set-allow-crimson', '--yes-i-really-mean-it'])


nit:
tasks/ceph.py has already implemented this:

@contextlib.contextmanager def check_enable_crimson(ctx, config): # enable crimson-osds if crimson log.info("check_enable_crimson: {}".format(is_crimson(config))) if is_crimson(config): cluster_name = config['cluster'] first_mon = teuthology.get_first_mon(ctx, config, cluster_name) (mon_remote,) = ctx.cluster.only(first_mon).remotes.keys() log.info('check_enable_crimson: setting set-allow-crimson') mon_remote.run( args=[ 'sudo', 'ceph', '--cluster', cluster_name, 'osd', 'set-allow-crimson', '--yes-i-really-mean-it' ] ) yield

can we reuse some of the logic above? Perhaps is_crimson can be reused

In the classic OSD test suite both ubuntu and centos builds are supported. However the crimson OSD only provides centos based builds so the same test layout cannot be reused directly. To enable cephadm testing with crimson, this change ports the relevant test suite to the `crimson-rados` directory. Instead of linking entire subdirectories (like .qa/suites/orch/cephadm/{osds,smoke,workunits,smoke-singlehost}), the crimson suite uses selective symlinks for the required yaml files and exclude ubuntu specific yaml files. Fixes: https://tracker.ceph.com/issues/71946 Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>

Matan-B

lgtm!

From looking at https://pulpito.ceph.com/moagrawa-2025-11-13_05:59:02-crimson-rados-main-distro-default-smithi/ we still have a few failures. We should make sure only stable tests are included in crimson-rados suite.

The failed jobs could be either resolved or moved to the crimson-rados-experimental suite.

Matan-B · 2025-11-13T09:11:59Z

qa/tasks/cephadm.py

                   check_status=False)

+        log.info('Bootstrapping:checking and enabling crimson if needed')
+        check_enable_crimson(ctx, config)


That's much cleaner!

Matan-B · 2025-11-13T09:17:04Z

@mohit84, Once the crimson-rados/cepadm suite is green we can merge

mohit84 · 2025-11-13T10:58:58Z

@mohit84, Once the crimson-rados/cepadm suite is green we can merge

I have checked after make a symlink number of teuthology jobs are not same similar to classic rados, i think we have to change the approach.

Matan-B

Can we change the default store used for testing crimson-rados/cephadm to SeaStore? Or add it as an option? It seems that cephadm suite is using bluestore only by default atm

github-actions · 2026-01-18T10:02:01Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

github-actions · 2026-02-17T10:14:33Z

This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution!

mohit84 requested a review from a team as a code owner November 12, 2025 11:23

github-actions bot added cephadm crimson tests labels Nov 12, 2025

mohit84 removed tests cephadm labels Nov 12, 2025

mohit84 requested review from Matan-B and athanatos November 12, 2025 11:23

Matan-B reviewed Nov 12, 2025

View reviewed changes

Matan-B added this to Crimson Nov 12, 2025

Matan-B moved this to In Progress in Crimson Nov 12, 2025

mohit84 force-pushed the cephadm_crimson branch from e521d25 to 7270800 Compare November 12, 2025 11:35

github-actions bot added cephadm tests labels Nov 12, 2025

Matan-B reviewed Nov 12, 2025

View reviewed changes

mohit84 force-pushed the cephadm_crimson branch from 7270800 to e900714 Compare November 13, 2025 05:55

mohit84 removed the status in Crimson Nov 13, 2025

mohit84 moved this to In Progress in Crimson Nov 13, 2025

Matan-B approved these changes Nov 13, 2025

View reviewed changes

Matan-B moved this from In Progress to Needs QA in Crimson Nov 13, 2025

Matan-B added the needs-crimson-qa label Nov 13, 2025

Matan-B moved this from Needs QA to In Progress in Crimson Nov 18, 2025

Matan-B removed the needs-crimson-qa label Nov 18, 2025

Matan-B reviewed Nov 19, 2025

View reviewed changes

Matan-B mentioned this pull request Dec 2, 2025

qa/config/crimson_qa_overrides: cephadm to use crimson binary #66458

Closed

14 tasks

github-actions bot added the stale label Jan 18, 2026

github-actions bot closed this Feb 17, 2026

Conversation

mohit84 commented Nov 12, 2025

Contribution Guidelines

Checklist

Uh oh!

Matan-B left a comment

Choose a reason for hiding this comment

Uh oh!

Matan-B Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

mohit84 Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

Matan-B commented Nov 12, 2025

Uh oh!

Matan-B Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Matan-B Nov 12, 2025

Choose a reason for hiding this comment

Uh oh!

mohit84 Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

Matan-B left a comment

Choose a reason for hiding this comment

Uh oh!

Matan-B Nov 13, 2025

Choose a reason for hiding this comment

Uh oh!

Matan-B commented Nov 13, 2025

Uh oh!

mohit84 commented Nov 13, 2025

Uh oh!

Matan-B left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Jan 18, 2026

Uh oh!

github-actions bot commented Feb 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Matan-B Nov 12, 2025 •

edited

Loading