Skip to content

[CCR] Auto follow pattern APIs adjustments#34518

Merged
martijnvg merged 7 commits intoelastic:masterfrom
martijnvg:ccr_auto_follow_api_adjustments
Oct 23, 2018
Merged

[CCR] Auto follow pattern APIs adjustments#34518
martijnvg merged 7 commits intoelastic:masterfrom
martijnvg:ccr_auto_follow_api_adjustments

Conversation

@martijnvg
Copy link
Copy Markdown
Member

@martijnvg martijnvg commented Oct 16, 2018

  • Changed the resource id of auto follow patterns to be a user defined name in all auto follow pattern crud APIs.
  • Which leader cluster to use is now specified in the body of an auto follow pattern.
  • Fail when an unfollowed leader index matches with two or more auto follow patterns. The unfollow leader index will then not be followed and the error can be seen in the auto follow stats API. The reason this validation is needed is, because multiple auto follow patters can be defined for a single leader cluster.

@martijnvg martijnvg added >non-issue :Distributed/CCR Issues around the Cross Cluster State Replication features labels Oct 16, 2018
@martijnvg martijnvg requested a review from jasontedor October 16, 2018 12:36
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-distributed

@martijnvg
Copy link
Copy Markdown
Member Author

run gradle build tests

if (randomBoolean()) {
putAutoFollowPatterns("my-pattern", new String[] {"logs-*", "transactions-*"});
} else {
putAutoFollowPatterns("my-pattern1", new String[] {"logs-*"});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

}

private void putAutoFollowPatterns(String... patterns) {
public void testConflictingPatterns() throws Exception {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if two patterns match an index, why is that a problem? we used to be able to do it before no?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So only one auto follow pattern would trigger the index following and when applying the second one would result in an error (same follow index already exists in follow cluster). Also which auto follow pattern gets applied first is undefined.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked at it more and I realized I got confused - with the new model we can still specify multiple patterns in a single auto follow config (is this a better name?). All good. Thanks for explaining.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with the new model we can still specify multiple patterns in a single auto follow config (is this a better name?)

Yes, we can still do that.

Copy link
Copy Markdown
Member

@jasontedor jasontedor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@martijnvg martijnvg merged commit 36baf38 into elastic:master Oct 23, 2018
martijnvg added a commit that referenced this pull request Oct 23, 2018
* Changed the resource id of auto follow patterns to be a user defined name
instead of being the leader cluster alias name.
* Fail when an unfollowed leader index matches with two or more auto follow patterns.
jasontedor added a commit to jasontedor/elasticsearch that referenced this pull request Oct 23, 2018
* master: (24 commits)
  ingest: better support for conditionals with simulate?verbose (elastic#34155)
  [Rollup] Job deletion should be invoked on the allocated task (elastic#34574)
  [DOCS] .Security index is never auto created (elastic#34589)
  CCR: Requires soft-deletes on the follower (elastic#34725)
  re-enable bwc tests (elastic#34743)
  Empty GetAliases authorization fix (elastic#34444)
  INGEST: Document Processor Conditional (elastic#33388)
  [CCR] Add total fetch time leader stat (elastic#34577)
  SQL: Support pattern against compatible indices (elastic#34718)
  [CCR] Auto follow pattern APIs adjustments (elastic#34518)
  [Test] Remove dead code from ExceptionSerializationTests (elastic#34713)
  A small typo in migration-assistance doc (elastic#34704)
  ingest: processor stats (elastic#34724)
  SQL: Implement IN(value1, value2, ...) expression. (elastic#34581)
  Tests: Add checks to GeoDistanceQueryBuilderTests (elastic#34273)
  INGEST: Rename Pipeline Processor Param. (elastic#34733)
  Core: Move IndexNameExpressionResolver to java time (elastic#34507)
  [DOCS] Force Merge: clarify execution and storage requirements (elastic#33882)
  TESTING.asciidoc fix examples using forbidden annotation (elastic#34515)
  SQL: Implement `CONVERT`, an alternative to `CAST` (elastic#34660)
  ...
kcm pushed a commit that referenced this pull request Oct 30, 2018
* Changed the resource id of auto follow patterns to be a user defined name
instead of being the leader cluster alias name.
* Fail when an unfollowed leader index matches with two or more auto follow patterns.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

:Distributed/CCR Issues around the Cross Cluster State Replication features >non-issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants