💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

🤖 GitHub comments

@jsoriano @hop-dev I think we need to quickly disucss the current spec/format for input packages before we merge this one. I want to see if we can clarify a few things we have issues with.

1. Is the format for system test folders the same now without datastreams?
2. How do we generate docs?
3. Where should we put build.yml?

Looking at the log integration package, it is missing several crucial pieces, so we cant use that as a reference.

I thought it would be quickest to discuss this quickly on slack, so we can get a good understanding of the intended format.

@jsoriano @hop-dev I think we need to quickly disucss the current spec/format for input packages before we merge this one. I want to see if we can clarify a few things we have issues with.

1. Is the format for system test folders the same now without datastreams?

It is mostly the same, yes. Regarding the spec intended format, you can see an input package as a single datastream.

2. How do we generate docs?

In principle it is also the same, with files under the docs directory. Though current helpers for data streams probably won't work now. I think we haven't added any input with documented fields yet, this would be something to improve if it doesn't work now.

3. Where should we put build.yml?

According to the spec, in the same place as in integration packages, in _dev/build/build.yml.

Looking at the log integration package, it is missing several crucial pieces, so we cant use that as a reference.

Are you missing other pieces apart of the ones you mention here?

I thought it would be quickest to discuss this quickly on slack, so we can get a good understanding of the intended format.

Maybe we can schedule a meeting, including also @hop-dev and @ishleenk17.

2. How do we generate docs?

We don't get the sample event in the docs using the helpers, unlike in integrations.
It doesn't follow the auto generations format. It has to be done manually.

These are some of the input packages already implemented:
Jolokia Input, SQL, Statsd.

All of them might not have system tests though.

Maybe we can schedule a meeting, including also @hop-dev and @ishleenk17.

Happy to do that.
I have shared some of my initial review comments. Let me know if a meeting is required as well.

@ishleenk17

Datastream name needs to be taken up as input from the user in your journald.yml.hbs file

Shouldn't we let elastic-package or fleet handle this? We already inject the UI elements to configure this, might as well also inject it into the config? Its easy to forget, because we have to remove the datastream settings from the manifest.yml, but keep it in the *.yml.hbs files.

@ishleenk17

Datastream name needs to be taken up as input from the user in your journald.yml.hbs file

Shouldn't we let elastic-package or fleet handle this? We already inject the UI elements to configure this, might as well also inject it into the config? Its easy to forget, because we have to remove the datastream settings from the manifest.yml, but keep it in the *.yml.hbs files.

Fleet might be the right place to handle this for the input package.
But, currently, it's not supported. So you might want to add it for now and raise this request with Fleet.

For input packages, we let the user do all the fields mappings through Kibana.
We don't do any mapping of the processed data under fields (like done in input.yml).
Only very generic fields should be mapped here.

@ishleenk17 I have some doubts regarding this suggestion. I understand we want let the users do the mapping of non-generic fields. However there are certain fields in input.yml that always appear in events (e.g. journald.pid) that might be interesting to keep.

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

Solved an issue with the mapping of event.created field.

Fixed at 892a487

Package journald - 1.0.0 containing this change is available at https://epr.elastic.co/search?package=journald