Bug #73630
closedobjectstore/test_fuse.sh: line 126: fusermount: command not found
0%
Description
Error in scrape.log:
Failure: Command failed (workunit test objectstore/test_fuse.sh) on smithi125 with status 127: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=430391ad9b2582054063ad3a7be9385a6839b835 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/objectstore/test_fuse.sh'
1 jobs: ['8552719']
suites: ['rados/objectstore/{backends/fusestore', 'supported-random-distro$/{centos_latest}}']
Error in teuthology.log:
2025-10-15T07:45:12.070 INFO:tasks.workunit.client.0.smithi125.stderr:+ grep -q 3 store_test_fuse_mnt/0.0_head/bitwise_hash_bits
2025-10-15T07:45:12.071 INFO:tasks.workunit.client.0.smithi125.stderr:+ grep -q 1fffffff store_test_fuse_mnt/0.0_head/bitwise_hash_end
2025-10-15T07:45:12.072 INFO:tasks.workunit.client.0.smithi125.stderr:+ test -d store_test_fuse_mnt/0.0_head/all
2025-10-15T07:45:12.073 INFO:tasks.workunit.client.0.smithi125.stderr:+ mkdir --mode 0003 store_test_fuse_mnt/0.1_head
2025-10-15T07:45:12.073 INFO:tasks.workunit.client.0.smithi125.stderr:+ grep -q 80000000 store_test_fuse_mnt/0.1_head/bitwise_hash_start
2025-10-15T07:45:12.075 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' bluestore = bluestore ']'
2025-10-15T07:45:12.075 INFO:tasks.workunit.client.0.smithi125.stderr:+ grep -q 3 store_test_fuse_mnt/0.1_head/bitwise_hash_bits
2025-10-15T07:45:12.076 INFO:tasks.workunit.client.0.smithi125.stderr:+ grep -q 9fffffff store_test_fuse_mnt/0.1_head/bitwise_hash_end
2025-10-15T07:45:12.077 INFO:tasks.workunit.client.0.smithi125.stderr:+ mkdir store_test_fuse_mnt/0.0_head/all/#0:00000000::::head#/
2025-10-15T07:45:12.078 INFO:tasks.workunit.client.0.smithi125.stderr:+ mkdir store_test_fuse_mnt/0.0_head/all/#0:10000000:::foo:head#/
2025-10-15T07:45:12.079 INFO:tasks.workunit.client.0.smithi125.stderr:+ '[' bluestore = bluestore ']'
2025-10-15T07:45:12.079 INFO:tasks.workunit.client.0.smithi125.stderr:+ expect_false mkdir store_test_fuse_mnt/0.0_head/all/#0:20000000:::bar:head#/
2025-10-15T07:45:12.079 INFO:tasks.workunit.client.0.smithi125.stderr:+ set -x
2025-10-15T07:45:12.080 INFO:tasks.workunit.client.0.smithi125.stderr:+ mkdir store_test_fuse_mnt/0.0_head/all/#0:20000000:::bar:head#/
2025-10-15T07:45:12.080 INFO:tasks.workunit.client.0.smithi125.stderr:mkdir: cannot create directory ‘store_test_fuse_mnt/0.0_head/all/#0:20000000:::bar:head#/’: Invalid argument
2025-10-15T07:45:12.081 INFO:tasks.workunit.client.0.smithi125.stderr:+ return 0
2025-10-15T07:45:12.081 INFO:tasks.workunit.client.0.smithi125.stderr:+ expect_false rmdir store_test_fuse_mnt/0.0_head
2025-10-15T07:45:12.081 INFO:tasks.workunit.client.0.smithi125.stderr:+ set -x
2025-10-15T07:45:12.081 INFO:tasks.workunit.client.0.smithi125.stderr:+ rmdir store_test_fuse_mnt/0.0_head
2025-10-15T07:45:12.087 INFO:tasks.workunit.client.0.smithi125.stderr:rmdir: failed to remove 'store_test_fuse_mnt/0.0_head': Directory not empty
2025-10-15T07:45:12.087 INFO:tasks.workunit.client.0.smithi125.stderr:+ return 0
2025-10-15T07:45:12.087 INFO:tasks.workunit.client.0.smithi125.stderr:+ rmdir store_test_fuse_mnt/0.0_head/all/#0:10000000:::foo:head#/
2025-10-15T07:45:12.088 INFO:tasks.workunit.client.0.smithi125.stderr:+ rmdir store_test_fuse_mnt/0.0_head/all/#0:00000000::::head#/
2025-10-15T07:45:12.089 INFO:tasks.workunit.client.0.smithi125.stderr:+ rmdir store_test_fuse_mnt/0.0_head
2025-10-15T07:45:12.095 INFO:tasks.workunit.client.0.smithi125.stderr:+ rmdir store_test_fuse_mnt/0.1_head
2025-10-15T07:45:12.096 INFO:tasks.workunit.client.0.smithi125.stderr:+ fusermount -u store_test_fuse_mnt
2025-10-15T07:45:12.096 INFO:tasks.workunit.client.0.smithi125.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/objectstore/test_fuse.sh: line 126: fusermount: command not found
2025-10-15T07:45:12.098 DEBUG:teuthology.orchestra.run:got remote process result: 127
2025-10-15T07:46:12.101 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: <Greenlet at 0x7f324c52b1a0: copy_file_to(<paramiko.ChannelFile from <paramiko.Channel 46 (E, <Logger tasks.workunit.client.0.smithi125.stderr (, None, False)>
2025-10-15T07:47:12.104 DEBUG:teuthology.orchestra.run:timed out waiting; will kill: <Greenlet at 0x7f324c52ad40: copy_file_to(<paramiko.ChannelFile from <paramiko.Channel 46 (E, <Logger tasks.workunit.client.0.smithi125.stdout (, None, False)>
2025-10-15T07:47:12.105 INFO:tasks.workunit:Stopping ['objectstore/test_fuse.sh'] on client.0...
2025-10-15T07:47:12.106 DEBUG:teuthology.orchestra.run.smithi125:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0
2025-10-15T07:47:12.417 ERROR:teuthology.run_tasks:Saw exception from tasks.
Updated by Shraddha Agrawal 5 months ago
- Subject changed from fusermount: command not found to objectstore/test_fuse.sh: line 126: fusermount: command not found
Updated by Laura Flores 5 months ago
/a/skanta-2025-10-09_23:38:36-rados-wip-bharath7-testing-2025-10-09-2128-distro-default-smithi/8543967
Updated by Radoslaw Zarzynski 5 months ago
- Project changed from RADOS to CephFS
I'm not entirely sure who's the proper owner of the bug. In short: test_fuse.sh is unable to fusermount. CephFS or build / packaging problem?
2025-10-15T07:45:12.096 INFO:tasks.workunit.client.0.smithi125.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/objectstore/test_fuse.sh: line 126: fusermount: command not found
Updated by Lee Sanders 4 months ago
/a/skanta-2025-11-01_01:12:28-rados-wip-bharath5-testing-2025-10-31-1454-distro-default-smithi/8578598
/a/skanta-2025-10-31_12:25:51-rados-wip-bharath5-testing-2025-10-31-1454-distro-default-smithi/8577865
Updated by Venky Shankar 4 months ago
I am not sure if this is even cephfs related. I see a similar issue in: /a/vshankar-2025-11-05_03:17:31-fs-wip-vshankar-testing-20251104.125650-debug-for-linus-default-smithi/8584782
2025-11-05T07:13:41.203 DEBUG:teuthology.orchestra.run.smithi101:> rpm -q fuse --qf '%{VERSION}-%{RELEASE}'
2025-11-05T07:13:41.226 INFO:teuthology.orchestra.run.smithi101.stdout:package fuse is not installed
2025-11-05T07:13:41.226 DEBUG:teuthology.orchestra.run:got remote process result: 1
2025-11-05T07:13:41.227 INFO:teuthology.nuke:Clearing teuthology firewall rules...
Updated by Laura Flores 4 months ago
/a/lflores-2025-11-19_18:47:12-rados-wip-lflores-testing-2-2025-11-19-1711-distro-default-smithi/8613335
Updated by Kamoltat (Junior) Sirivadhna 4 months ago
/a/skanta-2025-11-01_02:37:10-rados-wip-bharath4-testing-2025-10-31-1459-distro-default-smithi/
8578632
Updated by Laura Flores 4 months ago
/a/skanta-2025-11-22_15:10:56-rados-wip-lflores-testing-4-2025-11-21-1738-distro-default-smithi/8620123
Updated by Naveen Naidu 3 months ago
/a/skanta-2025-11-21_10:17:34-rados-wip-bharath11-testing-2025-11-21-0531-distro-default-smithi/8617784
Updated by Laura Flores about 2 months ago
/a/nmordech-2026-01-25_11:10:14-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/17026
Updated by Laura Flores about 2 months ago
/a/lflores-2026-01-26_23:21:06-rados-wip-yuri12-testing-2026-01-22-2045-distro-default-trial/19083
Updated by David Galloway about 2 months ago
Will be fixed by https://github.com/ceph/ceph-cm-ansible/pull/816
Updated by Laura Flores about 2 months ago
- Project changed from CephFS to Infrastructure
- Status changed from New to Resolved
Great! Moving this to the Infrastructure project. Marking as "Resolved" since the PR was merged, but let's reopen if it continues to be a problem.
Updated by Laura Flores about 1 month ago
/a/yuriw-2026-02-03_16:00:06-rados-wip-yuri4-testing-2026-02-02-2122-distro-default-trial/31522
Still leaving at resolved, since it's possible that this test occurred before the fix was merged.