You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This task is to implement rewriting and installation of ingest pipelines from packages.
For each input/dataset defined in a package:
The main ("entry") pipeline is defined in the manifest.yml of the input/dataset.
This main pipeline may contain references to other pipelines. The reference in the pipeline looks like this: {{IngestPipeline 'my-pipeline'}} and refers to a pipeline called my-pipeline.json in the same directory as the main pipeline in the package.
The main pipeline will be called like the index alias it refers to (e.g. logs-nginx-prod-access-aabbcc)
(40752 rewrite ingest pipeline #50627) Rewrite the content of the main pipeline, replacing every instance of {{IngestPipeline 'some-pipeline'}} with logs-nginx-prod-access-aabbcc-some-pipeline (the name of the main pipeline and the name of the referenced pipeline joined by - to one long string)
Install all referenced pipelines with their new names, e.g. logs-nginx-prod-access-aabbcc-some-pipeline
Finally install the rewritten main pipeline with the name logs-nginx-prod-access-aabbcc
Pipelines may also be yml-files. The same rules apply.
It is possible that an input/dataset doesn't have any ingest pipelines defined.
If the format ({{IngestPipeline 'my-pipeline'}}) turns out to be hard to parse we can still change it.
This task is to implement rewriting and installation of ingest pipelines from packages.
For each input/dataset defined in a package:
manifest.ymlof the input/dataset.{{IngestPipeline 'my-pipeline'}}and refers to a pipeline calledmy-pipeline.jsonin the same directory as the main pipeline in the package.logs-nginx-prod-access-aabbcc){{IngestPipeline 'some-pipeline'}}withlogs-nginx-prod-access-aabbcc-some-pipeline(the name of the main pipeline and the name of the referenced pipeline joined by-to one long string)logs-nginx-prod-access-aabbcc-some-pipelinelogs-nginx-prod-access-aabbccyml-files. The same rules apply.{{IngestPipeline 'my-pipeline'}}) turns out to be hard to parse we can still change it.