Skip to content

Regression in udev handling when partitions are created by ignition #1194

@jepio

Description

@jepio

Description

The following clc snippet (or the equivalent butane or ignition v3) doesn't work:

storage:
  disks:
  - device: /dev/vda
    partitions:
    - label: VAR
      number: 10
      start: '9GiB'
  filesystems:
  - name: var
    mount:
      device: /dev/disk/by-partlabel/VAR
      format: xfs
      label: var
  files:
  - filesystem: root
    path: /etc/fstab
    mode: 0644
    contents:
      inline: |
        /dev/disk/by-label/var /var xfs defaults 0 0

Ignition gets stuck waiting for device to appear and eventually times out and drops to an emergency shell.

Impact

Can't create new partitions in ignition (does it work for a secondary disk?)

Environment and steps to reproduce

  1. fetch flatcar stable disk image
  2. resize: qemu-img resize <disk.img> 20G
  3. run flatcar_production_qemu.sh with ignition

Expected behavior

Success

Additional information

This works on LTS and stable up to major 3374. 3510 and above fails like this:

[    5.780823] systemd[1]: Starting ignition-disks.service...
[    5.832946] ignition[718]: Ignition 2.14.0
[    5.838141] ignition[718]: Stage: disks
[    5.845725] ignition[718]: no configs at "/usr/lib/ignition/base.d"
[    5.854225] ignition[718]: no config dir at "/usr/lib/ignition/base.platform.d/qemu"
[    5.863847] ignition[718]: disks: createPartitions: op(1): [started]  waiting for devices [/dev/disk/by-id/virtio-primary-disk]
[    5.867880] ignition[718]: disks: createPartitions: op(1): [finished] waiting for devices [/dev/disk/by-id/virtio-primary-disk]
[    5.872746] ignition[718]: disks: createPartitions: created device alias for "/dev/disk/by-id/virtio-primary-disk": "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk" -> "/dev/vda"
[    5.883482] ignition[718]: disks: createPartitions: op(2): [started]  partitioning "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    5.893657] ignition[718]: disks: createPartitions: op(2): op(3): [started]  reading partition table of "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    5.902335] ignition[718]: disks: createPartitions: op(2): op(3): [finished] reading partition table of "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    5.906493] ignition[718]: disks: createPartitions: op(2): running sgdisk with options: [--pretend --new=10:18874368:+0 /run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk]
[    5.936738] ignition[718]: disks: createPartitions: op(2): running sgdisk with options: [--new=10:18874368:+0 --change-name=10:VAR /run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk]
[    5.940711] ignition[718]: disks: createPartitions: op(2): op(4): [started]  deleting 0 partitions and creating 1 partitions on "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    6.154788] BTRFS: device label OEM devid 1 transid 13 /dev/vda6 scanned by systemd-udevd (446)
[    6.958227] ignition[718]: disks: createPartitions: op(2): op(4): [finished] deleting 0 partitions and creating 1 partitions on "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    6.972495] ignition[718]: disks: createPartitions: op(2): op(5): [started]  waiting for triggered uevent
[    7.172562] ignition[718]: disks: createPartitions: op(2): op(5): [finished] waiting for triggered uevent
[    7.178744] ignition[718]: disks: createPartitions: op(2): [finished] partitioning "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    7.186823] ignition[718]: disks: createFilesystems: op(6): [started]  waiting for devices [/dev/disk/by-partlabel/VAR]
[    7.194799] systemd-networkd[685]: eth0: Gained IPv6LL
[ESC[0mESC[0;31m*     ESC[0m] (1 of 2) Job dev-disk-by\x2dpartlab…vice/start running (2s / no limit)
ESCM^MESC[K[ESC[0;1;31m*ESC[0mESC[0;31m*    ESC[0m] (1 of 2) Job dev-disk-by\x2dpartlab…vice/start running (2s / no limit)
ESCM^MESC[K[ESC[0;31m*ESC[0;1;31m*ESC[0mESC[0;31m*   ESC[0m] (1 of 2) Job dev-disk-by\x2dpartlab…vice/start running (3s / no limit)
ESCM^MESC[K[ ESC[0;31m*ESC[0;1;31m*ESC[0mESC[0;31m*  ESC[0m] (2 of 2) Job ignition-disks.service/start running (9s / no limit)
ESCM^MESC[K[  ESC[0;31m*ESC[0;1;31m*ESC[0mESC[0;31m* ESC[0m] (2 of 2) Job ignition-disks.service/start running (10s / no limit)
ESCM^MESC[K[   ESC[0;31m*ESC[0;1;31m*ESC[0mESC[0;31m*ESC[0m] (2 of 2) Job ignition-disks.service/start running (10s / no limit)
ESCM^MESC[K[    ESC[0;31m*ESC[0;1;31m*ESC[0m] (1 of 2) Job dev-disk-by\x2dpartlab…vice/start running (5s / no limit)
ESCM^MESC[K[     ESC[0;31m*ESC[0m] (1 of 2) Job dev-disk-by\x2dpartlab…vice/start running (6s / no limit)
ESCM^MESC[K[    ESC[0;31m*ESC[0;1;31m*ESC[0m] (1 of 2) Job dev-disk-by\x2dpartlab…vice/start running (6s / no limit)
ESCM^MESC[K[   ESC[0;31m*ESC[0;1;31m*ESC[0mESC[0;31m*ESC[0m] (2 of 2) Job ignition-disks.service/start running (13s / no limit)
ESCM^MESC[K[  ESC[0;31m*ESC[0;1;31m*ESC[0mESC[0;31m* ESC[0m] (2 of 2) Job ignition-disks.service/start running (13s / no limit)

3374 works, the output looks like this:

         Starting ESC[0;1;39mIgnition (disks)ESC[0m...
[    6.825113] systemd[1]: Starting Ignition (disks)...
[    6.869640] ignition[520]: Ignition 2.14.0
[    6.872458] ignition[520]: Stage: disks
[    6.874437] ignition[520]: no configs at "/usr/lib/ignition/base.d"
[    6.878769] ignition[520]: no config dir at "/usr/lib/ignition/base.platform.d/qemu"
[    6.881837] ignition[520]: disks: createPartitions: op(1): [started]  waiting for devices [/dev/disk/by-id/virtio-primary-disk]
[    6.903665] ignition[520]: disks: createPartitions: op(1): [finished] waiting for devices [/dev/disk/by-id/virtio-primary-disk]
[    6.922524] ignition[520]: disks: createPartitions: created device alias for "/dev/disk/by-id/virtio-primary-disk": "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk" -> "/dev/vda"
[    6.929480] ignition[520]: disks: createPartitions: op(2): [started]  partitioning "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    6.947332] ignition[520]: disks: createPartitions: op(2): op(3): [started]  reading partition table of "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    6.953799] ignition[520]: di[    7.194624]  vda: vda1 vda2 vda3 vda4 vda6 vda7 vda9
sks: createPartitions: op(2): op(3): [finished] reading partition table of "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    6.966963] ignition[520]: disks: createPartitions: op(2): running sgdisk with options: [--pretend --new=10:18874368:+0 /run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk]
[    6.985414] ignition[520]: disks: createPartitions: op(2): running sgdisk with options: [--new=10:18874368:+0 --change-name=10:VAR /run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk]
[    6.994395] ignition[520]: disks: createPartitions: op(2): op(4): [started]  deleting 0 partitions and creating 1 partitions on "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    7.454212] systemd-networkd[347]: eth0: Gained IPv6LL
[    8.248506]  vda: vda1 vda2 vda3 vda4 vda6 vda7 vda9 vda10
[    8.039379] ignition[520]: disks: createPartitions: op(2): op(4): [finished] deleting 0 partitions and creating 1 partitions on "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    8.050373] ignition[520]: disks: createPartitions: op(2): op(5): [started]  waiting for triggered uevent
[    8.385328] ignition[520]: disks: createPartitions: op(2): op(5): [finished] waiting for triggered uevent
[    8.392767] ignition[520]: disks: createPartitions: op(2): [finished] partitioning "/run/ignition/dev_aliases/dev/disk/by-id/virtio-primary-disk"
[    8.402272] ignition[520]: disks: createFilesystems: op(6): [started]  waiting for devices [/dev/disk/by-partlabel/VAR]
[    8.442391] ignition[520]: disks: createFilesystems: op(6): [finished] waiting for devices [/dev/disk/by-partlabel/VAR]
[    8.451259] ignition[520]: disks: createFilesystems: created device alias for "/dev/disk/by-partlabel/VAR": "/run/ignition/dev_aliases/dev/disk/by-partlabel/VAR" -> "/dev/vda10"
[    8.458365] ignition[520]: disks: createFilesystems: op(7): [started]  determining filesystem type of "/dev/disk/by-partlabel/VAR"
[    8.484467] ignition[520]: disks: createFilesystems: op(7): [finished] determining filesystem type of "/dev/disk/by-partlabel/VAR"
[    8.491105] ignition[520]: disks: createFilesystems: found  filesystem at "/dev/disk/by-partlabel/VAR" with uuid "" and label ""
[    8.493757] ignition[520]: disks: createFilesystems: op(8): [started]  wiping filesystem signatures from "/run/ignition/dev_aliases/dev/disk/by-partlabel/VAR"
[    8.497372] ignition[520]: disks: createFilesystems: op(8): [finished] wiping filesystem signatures from "/run/ignition/dev_aliases/dev/disk/by-partlabel/VAR"
[    8.502561] ignition[520]: disks: createFilesystems: op(9): [started]  creating "xfs" filesystem on "/run/ignition/dev_aliases/dev/disk/by-partlabel/VAR"
[    8.592445] ignition[520]: disks: createFilesystems: op(9): [finished] creating "xfs" filesystem on "/run/ignition/dev_aliases/dev/disk/by-partlabel/VAR"
[    8.598342] ignition[520]: disks: createFilesystems: op(a): [started]  waiting for triggered uevent
[    8.634420] ignition[520]: disks: createFilesystems: op(a): [finished] waiting for triggered uevent
[    8.639457] ignition[520]: disks: disks passed
[ESC[0;32m  OK  ESC[0m] Finished ESC[0;1;39mIgnition (disks)ESC[0m.

Please add any information here that does not fit the above format.

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions