Pinging @elastic/integrations (Team:Integrations)

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

we need some kind of system that populates a test directory inside the agent container with a handful of files that the data streams can read from

Could you elaborate on this? Are you talking about /proc&/sys folders, or is this something else?

we need some kind of system that populates a test directory inside the agent container with a handful of files that the data streams can read from

For system tests of metrics so far I think that we are only checking that metrics are collected without errors, we are not checking their values. For this case it may be also enough with checking that it collects any data, so the actual /proc and /sys filesystems available in the agent container would serve.

For logs you can use ${SERVICE_LOGS_DIR}.

Could you elaborate on this? Are you talking about /proc&/sys folders, or is this something else?
@exekias

Yes, along with a few other things. Depending on the OS config, some procfs files (such as RAID) won't contain any useful information for the metricset to parse. Some, such as cgroup files, won't exist at all inside a container. I spent some time hacking at the container orchestration that exists now to do that, but it was awfully ugly and I decided not to. At minimum, it would be nice to omit the "auxillary" container that elastic-package test system seems to require, as it's not actually needed here.

For system tests of metrics so far I think that we are only checking that metrics are collected without errors, we are not checking their values. For this case it may be also enough with checking that it collects any data, so the actual /proc and /sys filesystems available in the agent container would serve.

I'm very much onboard with this for the initial batch of tests (this PR). It won't be testing the full package but many parts of it are still valid.

On top of that (maybe later), we could also have a testdata folder mounted into the agent, the same way we have one for logs, right? This way you could mock a raid file structure and configure the agent to retrieve it, for example.

@mtojek @jsoriano any thoughts on the need for a dummy container when there is no need for it? I'm not particularly concerned with its need by I see @fearful-symmetry point

This is a specific use case where you don't need to deploy anything. Temporarily you can use a lightweight/micro/dummy image.

Definitely not a blocker, but could be considered as a feature. Please open an issue in elastic-package for this.

Alright, added the ECS fields to the other data streasm. @mtojek / @exekias I agree this isn't a blocker, just wanted to make the shortcomings known here. I'll open an issue on elastic-package.