SSSD startup failure with 4487.0.0 #1919
Description
Description
I was trying out the newest 4487.0.0 release and noticed that SSSD fails to start with the following error:
[sssd] [ldb] (0x0010): Unable to find backend for '/var/lib/sss/db/config.ldb' - do you need to set LDB_MODULES_PATH?
SSSD works just fine with the previous alpha, works with current beta and current stable. There wasn't any SSSD update in the newest release, so I suspect a deeper problem.
My best guess is, that this is related to some changes in Samba 4.21.0 in regards to LDB, which seems to be a dependency of SSSD.
Impact
SSSD doesn't start. This will impact users that configure LDAP/AD login in their flatcar installation.
Environment and steps to reproduce
- Start flatcar qemu image
- Start sssd with
sudo sssd - sssd fails immediately with the following errors:
sudo ./flatcar_production_qemu.sh -nographic
[...]
Flatcar Container Linux by Kinvolk alpha 4487.0.0 for QEMU
core@localhost ~ $ sudo sssd
[sssd] [ldb] (0x0010): Unable to find backend for '/var/lib/sss/db/config.ldb' - do you need to set LDB_MODULES_PATH?
[sssd] [confdb_init] (0x0010): Unable to open config database [/var/lib/sss/db/config.ldb]
[sssd] [confdb_setup] (0x0010): The confdb initialization failed [5]: Input/output error
[sssd] [load_configuration] (0x0010): Unable to setup ConfDB [5]: Input/output error
[sssd] [main] (0x0010): SSSD couldn't load the configuration database [5]: Input/output error
core@localhost ~ $ sudo ls -lah /var/lib/sss/db
total 8.0K
drwxr-xr-x. 2 root root 4.0K Oct 14 13:51 .
drwxr-xr-x. 10 root root 4.0K Oct 14 13:51 ..
The relevant part is probably the first line, where it can't find a proper backend and the rest are follow-up errors.
Expected behavior
Even without explicit sssd config, sssd should create /var/lib/sss/db/config.ldb and then fail later because the config is incomplete:
sudo ./flatcar_production_qemu.sh -nographic
[...]
Flatcar Container Linux by Kinvolk beta 4459.1.0 for QEMU
core@localhost ~ $ sudo sssd
[sssd] [sss_ini_add_snippets] (0x0020): Config merge error: Directory /etc/sssd/conf.d does not exist.
[sssd] [confdb_expand_app_domains] (0x0010): No domains configured, fatal error!
[sssd] [get_monitor_config] (0x0010): Failed to expand application domains
[sssd] [confdb_get_domains] (0x0020): No domains configured, fatal error!
[sssd] [get_monitor_config] (0x0010): No domains configured.
[sssd] [main] (0x0010): SSSD couldn't load the configuration database [1432158246]: No domain is enabled
core@localhost ~ $ sudo ls -lah /var/lib/sss/db
total 1.3M
drwxr-xr-x. 2 root root 4.0K Oct 14 14:01 .
drwxr-xr-x. 10 root root 4.0K Oct 14 11:28 ..
-rw-------. 1 root root 1.3M Oct 14 14:01 config.ldb
With a proper config, sssd should obviously start and run as a proper systemd unit, but I've kept it short for reproducibility.
Ideally it would look something like this:
Oct 14 14:14:58 node-1 systemd[1]: Starting sssd.service - System Security Services Daemon...
Oct 14 14:14:58 node-1 sssd[156914]: Starting up
Oct 14 14:14:58 node-1 sssd_be[156915]: Starting up
Oct 14 14:14:58 node-1 sssd_ssh[156919]: Starting up
Oct 14 14:14:58 node-1 sssd_nss[156916]: Starting up
Oct 14 14:14:58 node-1 sssd_sudo[156918]: Starting up
Oct 14 14:14:58 node-1 sssd_pam[156917]: Starting up
Oct 14 14:14:58 node-1 systemd[1]: Started sssd.service - System Security Services Daemon.
Additional information
To support my theory with the samba change, I checked what files sssd tries to open.
On a working release, sssd opens "/usr/lib64/samba/ldb/ldb.so".
Flatcar Container Linux by Kinvolk beta 4459.1.0 for QEMU
core@localhost ~ $ sudo strace -e trace=openat sssd 2>&1 | grep "/ldb.so"
openat(AT_FDCWD, "/usr/lib64/samba/ldb/ldb.so", O_RDONLY|O_CLOEXEC) = 4
core@localhost ~ $ sudo ls -lah /usr/lib64/samba/ldb
total 208K
drwxr-xr-x. 1 root root 174 Oct 12 23:39 .
drwxr-xr-x. 1 root root 5.6K Oct 12 23:39 ..
-rwxr-xr-x. 1 root root 15K Oct 12 22:04 asq.so
-rwxr-xr-x. 1 root root 15K Oct 12 22:04 ldb.so
-rwxr-xr-x. 1 root root 76K Oct 12 22:32 memberof.so
-rwxr-xr-x. 1 root root 15K Oct 12 22:04 paged_searches.so
-rwxr-xr-x. 1 root root 19K Oct 12 22:04 rdn_name.so
-rwxr-xr-x. 1 root root 15K Oct 12 22:04 sample.so
-rwxr-xr-x. 1 root root 15K Oct 12 22:04 server_sort.so
-rwxr-xr-x. 1 root root 15K Oct 12 22:04 skel.so
-rwxr-xr-x. 1 root root 15K Oct 12 22:04 tdb.so
With 4487.0.0 this file/library doesn't exist:
Flatcar Container Linux by Kinvolk alpha 4487.0.0 for QEMU
core@localhost ~ $ sudo find / -name "ldb.so"
<nothing>
core@localhost ~ $ sudo ls -lah /usr/lib64/samba/ldb
total 76K
drwxr-xr-x. 1 root root 22 Oct 13 04:26 .
drwxr-xr-x. 1 root root 5.9K Oct 13 04:26 ..
-rwxr-xr-x. 1 root root 76K Oct 13 03:28 memberof.so