ceph-volume: avoid unnecessary subprocess calls by guits · Pull Request #46766 · ceph/ceph

guits · 2022-06-21T12:01:25Z

These calls are slowing down ceph-volume inventory.
Especially because of the class ceph_volume.util.device.Devices.
It calls the class ceph_volume.util.device.Device for each device found
on the host which calls many times the binaries lsblk, pvs, vgs, lvs itself.
We can make only one call in Devices() to gather all details and use them during the whole runtime

current implementation:

        1    0.000    0.000    0.892    0.892 device.py:35(__init__) (class Devices)
        8    0.001    0.000    0.853    0.107 device.py:151(_parse)
       56    0.002    0.000    0.882    0.016 process.py:154(call)
        8    0.001    0.000    0.245    0.031 lvm.py:1099(get_lvs)
        8    0.000    0.000    0.026    0.003 disk.py:231(lsblk)
        8    0.000    0.000    0.435    0.054 device.py:278(_set_lvm_membership)
        1    0.000    0.000    0.885    0.885 device.py:38(<listcomp>) (multiple calls to Device() class)
      8/5    0.000    0.000    0.885    0.177 device.py:92(__init__) (class Device)

>>> timeit.timeit('Inventory([]).main()', setup='from ceph_volume.inventory import Inventory', number=1)

Device Path               Size         rotates available Model name
/dev/sdb                  200.00 GB    True    True      QEMU HARDDISK
/dev/sda                  200.00 GB    True    False     QEMU HARDDISK
/dev/sdc                  200.00 GB    True    False     QEMU HARDDISK
/dev/sdd                  200.00 GB    True    False     QEMU HARDDISK
/dev/vda                  11.00 GB     True    False
0.9309048530412838
>>>

new approach:

        1    0.000    0.000    0.253    0.253 device.py:35(__init__) (class Devices)
        5    0.000    0.000    0.144    0.029 device.py:167(_parse)
       21    0.001    0.000    0.246    0.012 process.py:154(call)
        1    0.000    0.000    0.032    0.032 lvm.py:1110(get_lvs)
        1    0.000    0.000    0.005    0.005 disk.py:236(lsblk_all)
        5    0.000    0.000    0.062    0.012 device.py:309(_set_lvm_membership)
        1    0.000    0.000    0.179    0.179 device.py:41(<listcomp>) (multiple calls to Device() class)
        5    0.000    0.000    0.179    0.036 device.py:99(__init__) (class Device)

>>> timeit.timeit('Inventory([]).main()', setup='from ceph_volume.inventory import Inventory', number=1)

Device Path               Size         rotates available Model name
/dev/sdb                  200.00 GB    True    True      QEMU HARDDISK
/dev/sda                  200.00 GB    True    False     QEMU HARDDISK
/dev/sdc                  200.00 GB    True    False     QEMU HARDDISK
/dev/sdd                  200.00 GB    True    False     QEMU HARDDISK
/dev/vda                  11.00 GB     True    False
0.2486933789914474
>>>

Signed-off-by: Guillaume Abrioux gabrioux@redhat.com

guits · 2022-06-23T12:31:42Z

jenkins test ceph-volume tox

guits · 2022-06-24T04:09:12Z

jenkins test api

guits · 2022-06-24T16:42:56Z

jenkins test ceph-volume tox

This name is misleading, let's rename it for something clearer. Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

guits · 2022-06-27T14:03:32Z

jenkins test ceph-volume tox

guits · 2022-06-28T06:51:54Z

jenkins test ceph-volume tox

These calls are slowing down `ceph-volume inventory`. Especially because of the class `ceph_volume.util.device.Devices`. It calls the class `ceph_volume.util.device.Device` for each device found on the host which calls many times the binaries `lsblk`, `pvs`, `vgs`, `lvs` itself. We can make only one call in `Devices()` to gather all details and use them during the whole runtime current implementation: ``` 1 0.000 0.000 0.892 0.892 device.py:35(__init__) (class Devices) 8 0.001 0.000 0.853 0.107 device.py:151(_parse) 56 0.002 0.000 0.882 0.016 process.py:154(call) 8 0.001 0.000 0.245 0.031 lvm.py:1099(get_lvs) 8 0.000 0.000 0.026 0.003 disk.py:231(lsblk) 8 0.000 0.000 0.435 0.054 device.py:278(_set_lvm_membership) 1 0.000 0.000 0.885 0.885 device.py:38(<listcomp>) (multiple calls to Device() class) 8/5 0.000 0.000 0.885 0.177 device.py:92(__init__) (class Device) >>> timeit.timeit('Inventory([]).main()', setup='from ceph_volume.inventory import Inventory', number=1) Device Path Size rotates available Model name /dev/sdb 200.00 GB True True QEMU HARDDISK /dev/sda 200.00 GB True False QEMU HARDDISK /dev/sdc 200.00 GB True False QEMU HARDDISK /dev/sdd 200.00 GB True False QEMU HARDDISK /dev/vda 11.00 GB True False 0.9309048530412838 >>> ``` new approach: ``` 1 0.000 0.000 0.253 0.253 device.py:35(__init__) (class Devices) 5 0.000 0.000 0.144 0.029 device.py:167(_parse) 21 0.001 0.000 0.246 0.012 process.py:154(call) 1 0.000 0.000 0.032 0.032 lvm.py:1110(get_lvs) 1 0.000 0.000 0.005 0.005 disk.py:236(lsblk_all) 5 0.000 0.000 0.062 0.012 device.py:309(_set_lvm_membership) 1 0.000 0.000 0.179 0.179 device.py:41(<listcomp>) (multiple calls to Device() class) 5 0.000 0.000 0.179 0.036 device.py:99(__init__) (class Device) >>> timeit.timeit('Inventory([]).main()', setup='from ceph_volume.inventory import Inventory', number=1) Device Path Size rotates available Model name /dev/sdb 200.00 GB True True QEMU HARDDISK /dev/sda 200.00 GB True False QEMU HARDDISK /dev/sdc 200.00 GB True False QEMU HARDDISK /dev/sdd 200.00 GB True False QEMU HARDDISK /dev/vda 11.00 GB True False 0.2486933789914474 >>> ``` Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

guits · 2022-06-28T08:53:06Z

jenkins test ceph-volume tox

guits · 2022-06-28T14:36:59Z

http://pulpito.front.sepia.ceph.com/gabrioux-2022-06-29_04:51:15-orch:cephadm-wip-guits-testing-2022-06-28-0505-distro-default-smithi/

pcuzner · 2022-06-28T22:36:55Z

Looks good. I tested on a physical machine with a bunch of ssd/nvme drives

Before

[ceph: root@f12-h09-000-1029u /]# time ceph-volume inventory                                                                                                                                                                                 

Device Path               Size         rotates available Model name
/dev/sdb                  894.25 GB    False   True      INTEL SSDSC2KB96
/dev/sdc                  894.25 GB    False   True      INTEL SSDSC2KB96
/dev/sdd                  894.25 GB    False   True      INTEL SSDSC2KB96
/dev/sde                  894.25 GB    False   True      INTEL SSDSC2KB96
/dev/nvme0n1              894.25 GB    False   False     SAMSUNG MZ1LW960HMJP-00003
/dev/nvme1n1              894.25 GB    False   False     SAMSUNG MZQLB960HAJR-00007
/dev/nvme2n1              894.25 GB    False   False     SAMSUNG MZQLB960HAJR-00007
/dev/nvme3n1              447.12 GB    False   False     Linux
/dev/nvme3n2              447.12 GB    False   False     Linux
/dev/nvme3n3              447.12 GB    False   False     Linux
/dev/nvme3n4              447.12 GB    False   False     Linux
/dev/nvme3n5              447.12 GB    False   False     Linux
/dev/nvme3n6              447.12 GB    False   False     Linux
/dev/sda                  447.13 GB    False   False     INTEL SSDSC2KB48

real    0m3.559s
user    0m0.931s
sys     0m0.957s

After

(ceph-volume-testing) [root@f12-h09-000-1029u inventory]# time ceph-volume inventory                                                                                                                                                          
                                                                                                                                                                                                                                              
Device Path               Size         rotates available Model name                                                                                                                                                                           
/dev/sdb                  894.25 GB    False   True      INTEL SSDSC2KB96                                                                                                                                                                     
/dev/sdc                  894.25 GB    False   True      INTEL SSDSC2KB96                                                                                                                                                                     
/dev/sdd                  894.25 GB    False   True      INTEL SSDSC2KB96                                                                                                                                                                     
/dev/sde                  894.25 GB    False   True      INTEL SSDSC2KB96                                                                                                                                                                     
/dev/nvme0n1              894.25 GB    False   False     SAMSUNG MZ1LW960HMJP-00003                                                                                                                                                           
/dev/nvme1n1              894.25 GB    False   False     SAMSUNG MZQLB960HAJR-00007                                                                                                                                                           
/dev/nvme2n1              894.25 GB    False   False     SAMSUNG MZQLB960HAJR-00007                                                                                                                                                           
/dev/nvme3n1              447.12 GB    False   False     Linux                                                                                                                                                                                
/dev/nvme3n2              447.12 GB    False   False     Linux                                                                                                                                                                                
/dev/nvme3n3              447.12 GB    False   False     Linux                                                                                                                                                                                
/dev/nvme3n4              447.12 GB    False   False     Linux
/dev/nvme3n5              447.12 GB    False   False     Linux
/dev/nvme3n6              447.12 GB    False   False     Linux
/dev/sda                  447.13 GB    False   False     INTEL SSDSC2KB48

real    0m1.086s
user    0m0.424s
sys     0m0.295s
(ceph-volume-testing) [root@f12-h09-000-1029u inventory]# deactivate

adk3798 · 2022-07-01T18:20:39Z

https://pulpito.ceph.com/adking-2022-07-01_05:11:14-orch:cephadm-wip-adk-testing-2022-06-30-2110-distro-default-smithi/

4 Failed tests from

Command failed on smithi162 with status 123: "find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip --"

and 1 dead test from

{'smithi050.front.sepia.ceph.com': {'attempts': 12, 'censored': "the output has been hidden due to the fact that 'no_log: true' was specified for this result", 'changed': True}}

I believe these are all infra issues and I did see these same failures in another run with different PRs (and notably, not on the same tests) in
https://pulpito.ceph.com/adking-2022-07-01_05:08:10-orch:cephadm-wip-adk3-testing-2022-06-30-2111-distro-default-smithi/

Quite confident the failures are unrelated to PRs in the run, so I think we're good here.

adk3798

LGTM

guits added the ceph-volume label Jun 21, 2022

guits requested a review from a team as a code owner June 21, 2022 12:01

guits force-pushed the util-device-refactor branch 2 times, most recently from 8d0e337 to 9c049f0 Compare June 21, 2022 12:43

guits marked this pull request as draft June 21, 2022 12:43

guits force-pushed the util-device-refactor branch 7 times, most recently from 7a45989 to c5dae72 Compare June 23, 2022 12:31

guits marked this pull request as ready for review June 23, 2022 12:36

guits added the wip-guits-testing label Jun 23, 2022

adk3798 added the wip-adk-testing label Jun 23, 2022

guits force-pushed the util-device-refactor branch from c5dae72 to b849773 Compare June 23, 2022 16:44

guits force-pushed the util-device-refactor branch 2 times, most recently from 9a8f54b to f457913 Compare June 24, 2022 16:40

guits force-pushed the util-device-refactor branch from f457913 to d720fad Compare June 27, 2022 08:28

ceph-volume: rename _get_pv_paths() in util/device.py

325b6f5

This name is misleading, let's rename it for something clearer. Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>

guits force-pushed the util-device-refactor branch from d720fad to 15c9265 Compare June 27, 2022 14:01

guits force-pushed the util-device-refactor branch from 15c9265 to cb79153 Compare June 28, 2022 05:04

guits force-pushed the util-device-refactor branch from cb79153 to bea9f4b Compare June 28, 2022 08:53

adk3798 approved these changes Jul 1, 2022

View reviewed changes

guits merged commit a2a3897 into ceph:main Jul 5, 2022

guits deleted the util-device-refactor branch July 5, 2022 13:11

guits removed the wip-guits-testing label Jul 5, 2022

adk3798 mentioned this pull request Jul 13, 2022

quincy: ceph-volume: avoid unnecessary subprocess calls #46968

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ceph-volume: avoid unnecessary subprocess calls#46766

ceph-volume: avoid unnecessary subprocess calls#46766
guits merged 2 commits intoceph:mainfrom
guits:util-device-refactor

guits commented Jun 21, 2022 •

edited

Loading

Uh oh!

guits commented Jun 23, 2022

Uh oh!

guits commented Jun 24, 2022

Uh oh!

guits commented Jun 24, 2022

Uh oh!

guits commented Jun 27, 2022

Uh oh!

guits commented Jun 28, 2022

Uh oh!

guits commented Jun 28, 2022

Uh oh!

guits commented Jun 28, 2022 •

edited

Loading

Uh oh!

pcuzner commented Jun 28, 2022

Uh oh!

adk3798 commented Jul 1, 2022

Uh oh!

adk3798 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

guits commented Jun 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

guits commented Jun 23, 2022

Uh oh!

guits commented Jun 24, 2022

Uh oh!

guits commented Jun 24, 2022

Uh oh!

guits commented Jun 27, 2022

Uh oh!

guits commented Jun 28, 2022

Uh oh!

guits commented Jun 28, 2022

Uh oh!

guits commented Jun 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pcuzner commented Jun 28, 2022

Uh oh!

adk3798 commented Jul 1, 2022

Uh oh!

adk3798 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

guits commented Jun 21, 2022 •

edited

Loading

guits commented Jun 28, 2022 •

edited

Loading