Skip to content

[Feature] Cross Cluster Replication based on segment replication  #3020

@krishna-ggk

Description

@krishna-ggk

Is your feature request related to a problem?
Currently CCR implements logical document replication where in operations from translog are replayed onto follower cluster. However with segment replication coming into OpenSearch core, basing CCR on it has potential to bring in huge benefits such as speeding up replication, reduced memory/CPU etc.

What solution would you like?
Implement CCR based on segment replication.

What alternatives have you considered?
Haven't considered really - but there is an RFC for pluggable translog. It would be good to investigate if there are any strong reasons for having logical replication (instead of physical segment replication) and if so consider if basing CCR on pluggable translog has any merits to the idea.

Do you have any additional context?
Need to ensure the RFC considers following

  1. Can it support filter replication? (Seems like we may be able to leverage FilterLeafReader ) Is it required?
  2. Can an external durable pluggable store like S3 be used instead of directly replicating?
  3. How would mapping changes etc be propagated?
  4. Would there be a migration path from previous implementation?

There is also ton of useful discussion on this topic at 2482

cc: @nknize @mikemccand

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementEnhancement or improvement to existing feature or requestlucene

    Type

    No type

    Projects

    Status

    New

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions