Based on the review item: s1monw@042ef8c#diff-c69f941ab536509edea2ed0715c9a0adR169
Right now changes that are made to pipelines via the put pipeline api are not immediately picked up by all ingest nodes. Each ingest node periodically polls the .ingest index in the background and if there are changes it updates the in-memory representation of the pipelines.
If changes made to a pipeline via the put pipeline api would also trigger that all ingest nodes update their pipelines then things becomes much more predictable. This process isn't heavy, so I think we shouldn't be concerned about performance if we trigger a reload after each call the put pipeline api.
This should make using node ingest easier and also would make testing easier. (no need to wrap assertions inside assertBusy(...) blocks).
If we were to add an internal pipeline refresh api then the put and delete pipeline apis would you invoke that after each modification. The internal refresh api should trigger a reload of all ingest nodes and return when all the reloads have been completed.
Based on the review item: s1monw@042ef8c#diff-c69f941ab536509edea2ed0715c9a0adR169
Right now changes that are made to pipelines via the put pipeline api are not immediately picked up by all ingest nodes. Each ingest node periodically polls the .ingest index in the background and if there are changes it updates the in-memory representation of the pipelines.
If changes made to a pipeline via the put pipeline api would also trigger that all ingest nodes update their pipelines then things becomes much more predictable. This process isn't heavy, so I think we shouldn't be concerned about performance if we trigger a reload after each call the put pipeline api.
This should make using node ingest easier and also would make testing easier. (no need to wrap assertions inside assertBusy(...) blocks).
If we were to add an internal pipeline refresh api then the put and delete pipeline apis would you invoke that after each modification. The internal refresh api should trigger a reload of all ingest nodes and return when all the reloads have been completed.