We need to make the overall system robust to differing environments. Specifically, the agent needs to be able to handle running in differing kernel and userland environments. To facilitate this, we've added a GetGuestDetails() API call. That allows the runtime to query details of the environment and the agent running inside the image before containers are launched in it.

WARNING: Many random ideas ahead 😄 ...

I'm wondering if it might make sense to mandate that all kernel configs provide the /proc/config.gz file [1]:

• https://github.com/torvalds/linux/blob/master/init/Kconfig#L508..L526

When enabled, that option would allow the agent to send back to the runtime all kernel configs enabled in the guest environment. The runtime could then decide how best to proceed.

It's currently very difficult to define what the "minimum Kata kernel" should provide but this would atleast provide us with the raw data to start investigating how to solve this issue.

Benefits

• Would allow easier debug as we could use standard tooling (grep, diff, etc) to compare known "good" environments with experimental / user-created ones that are experiencing issues.
• Would allow the runtime to record more useful logging information (potentially, rather a lot more! :)
• Might allow us to start defining a "minimum" set of functionality versus "add-ons" that should not be necessarily built into the kernel (they could be modules instead) [2].

Costs

• Slightly bigger kernel size

Alternative

Provide the .config file used to build the kernel package in the package itself as:

/usr/share/doc/kata-linux-container/vmlinu{xz}-${version}.container.config.gz

That would be simple enough to do but wouldn't guarantee that

Related

• Create Linux kernel configuration script #8
• Should we provide multiple guest kernel config files? #55
• create script to validate kernel options across architectures #56

/cc @grahamwhaley, @sboeuf, @jcvenegas.

[1] - If we really wanted to mandate it, the agent could check for this file and abort if not found 😄

[2] - What would be really neat is if the kernel could go one stage beyond providing /proc/config.gz and allow userland to be able to determine which CONFIG_* options *are currently loaded into the running kernel (in other words, exclude all CONFIG_* options that relate exclusively to kernel modules that are not currently loaded). That would allow us to create a fully modular kernel, run a set of tests that we feel "fully exercises" Kata capabilities and then simply run something like:

$ sudo diff /proc/config.gz /proc/config/loaded.gz

... to identify what minimum set of kernel config options" define a "Kata kernel". At that point, we could make those options builtins (as they are "fastpath") and all other options could either be discarded entirely or made into modules ("slowpath").