When trying to override any field from agent.download in elastic-agent.yml, the values from the configuration file are simply ignored. That makes the DownloadConfig struct from configuration.SettingsConfig never accepting any values other than the default ones.

I did a bit of digging and it seems to be related to the httpcommon.HTTPTransportSettings struct embedded into artifact.Config:

I managed to reproduce it by creating a new struct with httpcommon.HTTPTransportSettings embedded and with the same struct tags, then I experienced the same bug.

Here is an example on how to reproduce it:

1. In internal/pkg/agent/configuration/settings.go edit SettingsConfig to have a new field:

// SettingsConfig is an collection of agent settings configuration.
type SettingsConfig struct {
	DownloadConfig   *artifact.Config                `yaml:"download" config:"download" json:"download"`
	ProcessConfig    *process.Config                 `yaml:"process" config:"process" json:"process"`
	GRPC             *server.Config                  `yaml:"grpc" config:"grpc" json:"grpc"`
	RetryConfig      *retry.Config                   `yaml:"retry" config:"retry" json:"retry"`
	MonitoringConfig *monitoringCfg.MonitoringConfig `yaml:"monitoring" config:"monitoring" json:"monitoring"`
	LoggingConfig    *logger.Config                  `yaml:"logging,omitempty" config:"logging,omitempty" json:"logging,omitempty"`
	FooStruct        *FooStruct                      `yaml:"foo" config:"foo" json:"foo"`

	// standalone config
	Reload *ReloadConfig `config:"reload" yaml:"reload" json:"reload"`
	Path   string        `config:"path" yaml:"path" json:"path"`
}

type FooStruct struct {
	Foo                              string                      `yaml:"foo" config:"foo" json"foo"`
	Bar                              string                      `yaml:"bar" config:"bar" json:"bar"`
	Answer                           int                         `yaml:"answer" config:"answer" json:"answer"`
	httpcommon.HTTPTransportSettings `config:",ignore" yaml:"-"`
}

2. Edit the elastic-agent.yml adding something like:

agent.foo:
  foo: "bar"
  bar: "foo"
  answer: 42

3. Start the agent and check the value of SettingsConfig.FooConfig right after the yaml file parsed and merged with the default config here:

   elastic-agent/internal/pkg/agent/configuration/configuration.go

   Line 32 in 552a87d

   if err := cfg.Unpack(c); err != nil {

It will be empty (all zero values) even though cfg contains the correct values.

4. Removing the embedded httpcommon.HTTPTransportSettings fix the problem. However it is not feasible for the original Elastic-Agent configuration.

The bug is likely on github.com/elastic/go-ucfg, I'm reporting it here because it's affecting the Elastic-Agent and that's the way I managed to reproduce it. Also if somebody else faces this configuration issue, it's documented here.