Failure to parse OEM id causes cascading failures #16
Description
When migrating a host to Flatcar from Container Linux the failure to parse the embedded OEM id causes metadata setup to fail which then cascades to networking failure, ssh key failure, etc.
In this specific case there was no backwards compatibility to parse and understand namespaced coreos. values from the kernel command line:
Feb 05 18:06:45 redbeard-2gb-1-1 coreos-metadata[682]: Error: initialization
Feb 05 18:06:45 redbeard-2gb-1-1 coreos-metadata[682]: Caused by: Couldn't find flag 'flatcar.oem.id' in cmdline file (/proc/cmdline)
Feb 05 18:06:45 redbeard-2gb-1-1 systemd[1]: motdgen.service: Succeeded.
Feb 05 18:06:45 redbeard-2gb-1-1 systemd[1]: Started Generate /run/flatcar/motd.
Feb 05 18:06:45 redbeard-2gb-1-1 systemd[1]: coreos-metadata-sshkeys@core.service: Main process exited, code=exited, status=1/FAILURE
Feb 05 18:06:45 redbeard-2gb-1-1 systemd[1]: coreos-metadata-sshkeys@core.service: Failed with result 'exit-code'.
Feb 05 18:06:45 redbeard-2gb-1-1 systemd[1]: Failed to start Flatcar Metadata Agent (SSH Keys).
Feb 05 18:06:45 redbeard-2gb-1-1 systemd[1]: Dependency failed for Multi-User System.
Feb 05 18:06:45 redbeard-2gb-1-1 systemd[1]: multi-user.target: Job multi-user.target/start failed with result 'dependency'.
this host was booted with the following command line:
Feb 05 18:06:37 localhost kernel: Command line: BOOT_IMAGE=/coreos/vmlinuz-a mount.usr=/dev/mapper/usr verity.usr=PARTUUID=7130c94a-213a-4e5a-8e26-6cce9662f132 rootflags=rw mount.usrflags=ro consoleblank=0 root=LABEL=ROOT console=ttyS0,115200n8 console=tty0 coreos.oem.id=digitalocean verity.usrhash=757b2f0595a625a89b99b6fe7e8b59bd33b3e384fd90f538067a0b7b2bc09822
If the user can successfully boot the host and be able to modify partition 6 of the boot disk (the "OEM" partition) it is possible that the missing metadata expected by the Flatcar tooling can be added as a triage by appending set linux_append="$linux flatcar.oem.id=digitalocean" to the file /usr/share/oem/grub.cfg, that being said this requires advanced knowledge of the OS in general and may not be exhaustive of all changes needed to bring the host into a good working state:
$ cat /usr/share/oem/grub.cfg
# CoreOS GRUB settings
set oem_id="digitalocean"
set linux_append="$linux flatcar.oem.id=digitalocean"