Submission type

• Bug report
• Request for enhancement (RFE)

systemd version the issue has been seen with

231

Version 231 from a recent Ubuntu 'yakkety' pre-release ISO image. The bug was originally seen with Ubuntu 16.04's 229, and the code in question is unchanged since September 2015 or so. The Ubuntu bug is bug 1611945.

Used distribution

Ubuntu yakkety (in development), Ubuntu 16.04

Bug description

We have servers with external SATA disk enclosures, connected through (e)SATA port multiplier ports. On recent kernels and systemd versions, /dev/disk/by-path is not correctly populated with symlinks for all disks; instead we see a symlink to only one disk on each port. The direct cause for this is that udev is assigning the same ID_PATH name (and related other values) to all disks on the same port, and only one out of them can actually get the name. Specifically, all of the disks on a single port get the name of, eg, pci-0000:02:00.0-ata-1 (in this case, for all disks on the first port on the controller; on the second port, it's -ata-2, and so on).

Based on code inspection, the underlying issue is that handle_scsi_ata() in src/udev/udev-builtin-path_id.c is not aware that multiple disks can be behind a single ATA port. The only thing it puts in the ID_PATH is the ATA port number ('port_no'), which is the same for all disks behind the port.

Our machines previously did not have this issue in Ubuntu 12.04 because in 12.04, the kernel did not clearly identify these disks as ATA disks in their sysfs path. In 12.04, such a disk showed up with the sysfs path of /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:00.0/host8/target8:0:0/8:0:0:0/block/sdk. In 16.04's kernel, the same enclosure slot has the sysfs path of /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/0000:02:00.0/ata1/host0/target0:0:0/0:0:0:0/block/sda. The presence of '/ata' in this path triggers udev's ATA-specific handling code, whereas in 12.04 the disks were handled by the generic SCSI code.

In case of bug report: Expected behaviour you didn't see

All disks present in /dev/disk/by-path/.

In case of bug report: Unexpected behaviour you saw

Only one out of four disks on each port multiplier port being visible in /dev/disk/by-path.

In fact we can sometimes see inconsistencies in what disk gets what symlink. For instance, here is part of a ls -l:

lrwxrwxrwx 1 root root  9 Aug 11 16:32 pci-0000:02:00.0-ata-2 -> ../../sdi
lrwxrwxrwx 1 root root 10 Aug 11 16:32 pci-0000:02:00.0-ata-2-part1 -> ../../sdi1
lrwxrwxrwx 1 root root  9 Aug 11 16:32 pci-0000:02:00.0-ata-3 -> ../../sdk
lrwxrwxrwx 1 root root 10 Aug 11 16:32 pci-0000:02:00.0-ata-3-part1 -> ../../sdm1

Here -ata-2 and ata-2-part1 point to the same disk (sdi and sdi1), but -ata-3 and -ata-3-part1 point to different disks; sdk has won the contest to be -ata-3, but sdm1 has won for -ata-3-part1. This is not surprising since udev explicitly says the behavior of multiple devices being assigned the same name is unspecified, but it does show that there is a problem.

In case of bug report: Steps to reproduce the problem

Boot a system with SATA port multiplier support and multiple disks connected to the same port, and observe /dev/disk/by-path not having all of them.

Additional data

I have a certain amount of data gathered from systemd 231 on our machine; this includes things like 'udevadm info --export-db' output, 'udevadm test ' for some devices, and ls -lR of the PCI hierarchy for a controller (with multiple disks connected to multiple ports). I can attach some or much of that if desired. And if data from systemd 229 (16.04's version) is acceptable we can get basically any additional data that you want, since the machine is installed with that version currently. Just let me know what would be useful.

We are probably willing to temporarily run systemd versions with experimental patches/modifications to test fixes to the issue for you (especially given that you may not have easy access to hardware with SATA port multipliers and multiple disks behind a single port), although it is easiest if we can drop the modification into the Ubuntu 16.04 systemd source base.