We have some data streams that have duplicated pipelines because currently it is not possible to reference pipelines outside of the current data stream using the IngestPipeline tag.
Would be nice to extend it so it could be used in a similar way to:
{{ IngestPipeline nameOfPipeline }} for pipelines in the same data stream{{ IngestPipeline nameOfDatastream/nameOfPipeline }} for pipelines in other data streams of the same package
Additionally, even though maybe not as big of a need right now, something like {{ IngestPipeline nameOfPackage/versionOfPackage/nameOfDatastream/nameOfPipeline }} could be used to be able to reference any pipeline from any package. The version must be specified if we want to reference pipelines across packages to avoid breaking changes.
Another consideration is that if we include a pipeline from another data stream or package, we need to not only install it but also any other pipelines that it references, and manage the life cycle of these dependencies accordingly.
We have some data streams that have duplicated pipelines because currently it is not possible to reference pipelines outside of the current data stream using the
IngestPipelinetag.Would be nice to extend it so it could be used in a similar way to:
{{ IngestPipeline nameOfPipeline }}for pipelines in the same data stream{{ IngestPipeline nameOfDatastream/nameOfPipeline }}for pipelines in other data streams of the same packageAdditionally, even though maybe not as big of a need right now, something like
{{ IngestPipeline nameOfPackage/versionOfPackage/nameOfDatastream/nameOfPipeline }}could be used to be able to reference any pipeline from any package. The version must be specified if we want to reference pipelines across packages to avoid breaking changes.Another consideration is that if we include a pipeline from another data stream or package, we need to not only install it but also any other pipelines that it references, and manage the life cycle of these dependencies accordingly.