Add Channel Factory parameter to Translog by RajatGupta02 · Pull Request #18918 · opensearch-project/OpenSearch

RajatGupta02 · 2025-08-05T08:33:39Z

Description

Adds a channel factory parameter to translog constructor which was earlier always defaulted to FileChannel::open. This enables us to pass our own channel factory which was not possible by overriding otherwise.

Related Issues

Add translog encryption opensearch-storage-encryption#39

Check List

Functionality includes testing.
API changes companion pull request created, if applicable.
Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

RajatGupta02 · 2025-08-05T08:48:28Z

cc: @andrross @sachinpkale @kumargu @cwperks

github-actions · 2025-08-05T09:35:57Z

❌ Gradle check result for a311a41: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-05T12:13:02Z

❌ Gradle check result for 5bb9f00: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

cwperks · 2025-08-06T03:05:11Z

@RajatGupta02 please add a CHANGELOG entry for this. Please note that the CHANGELOG may be reset tomorrow post code-freeze.

github-actions · 2025-08-06T04:17:12Z

❕ Gradle check result for 5bb9f00: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

codecov · 2025-08-06T04:17:35Z

Codecov Report

❌ Patch coverage is 57.14286% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.93%. Comparing base (669ec8d) to head (4c8df69).
⚠️ Report is 9 commits behind head on main.

Files with missing lines	Patch %	Lines
...a/org/opensearch/index/translog/LocalTranslog.java	0.00%	2 Missing ⚠️
...n/java/org/opensearch/index/translog/Translog.java	80.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18918      +/-   ##
============================================
- Coverage     72.96%   72.93%   -0.03%     
- Complexity    69486    69512      +26     
============================================
  Files          5648     5648              
  Lines        319329   319334       +5     
  Branches      46194    46195       +1     
============================================
- Hits         232993   232922      -71     
- Misses        67516    67638     +122     
+ Partials      18820    18774      -46

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

github-actions · 2025-08-06T07:43:03Z

✅ Gradle check result for b258b88: SUCCESS

Signed-off-by: Rajat Gupta <gptrajat@amazon.com>

github-actions · 2025-08-26T15:44:33Z

✅ Gradle check result for 6a3eced: SUCCESS

Signed-off-by: Craig Perkins <cwperx@amazon.com>

github-actions · 2025-08-26T17:45:37Z

❌ Gradle check result for 51b082c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

server/src/main/java/org/opensearch/index/translog/Translog.java

Signed-off-by: Rajat Gupta <gptrajat@amazon.com>

github-actions · 2025-08-26T18:57:55Z

❌ Gradle check result for 4c8df69: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

github-actions · 2025-08-27T15:47:09Z

✅ Gradle check result for 4c8df69: SUCCESS

* Add overload for channelFactory Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Fix tests Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Add Changelog entry Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Fix conflicts Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <atri.jiit@gmail.com> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <andrross@amazon.com> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <atri.jiit@gmail.com> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Switch to CompletionStage Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG entry Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <andrebvandeven@gmail.com> Signed-off-by: Andre van de Ven <113951599+andrevandeven@users.noreply.github.com> Signed-off-by: Andre van de Ven <andrevdv@amazon.com> Co-authored-by: Andre van de Ven <andrevdv@amazon.com> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <anurag.rai@uber.com> Signed-off-by: Anurag Rai <91844619+anuragrai16@users.noreply.github.com> * Add Changelog entry Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Add secondary constructor Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Modify changelog Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Update changelog Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <gptrajat@amazon.com> --------- Signed-off-by: Rajat Gupta <gptrajat@amazon.com> Signed-off-by: Atri Sharma <atri.jiit@gmail.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Andre van de Ven <andrebvandeven@gmail.com> Signed-off-by: Andre van de Ven <113951599+andrevandeven@users.noreply.github.com> Signed-off-by: Andre van de Ven <andrevdv@amazon.com> Signed-off-by: Anurag Rai <anurag.rai@uber.com> Signed-off-by: Anurag Rai <91844619+anuragrai16@users.noreply.github.com> Co-authored-by: Rajat Gupta <gptrajat@amazon.com> Co-authored-by: Atri Sharma <atri.jiit@gmail.com> Co-authored-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Andre van de Ven <113951599+andrevandeven@users.noreply.github.com> Co-authored-by: Andre van de Ven <andrevdv@amazon.com> Co-authored-by: Anurag Rai <91844619+anuragrai16@users.noreply.github.com>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Fix tests Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Add Changelog entry Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Fix conflicts Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <atri.jiit@gmail.com> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <andrross@amazon.com> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <atri.jiit@gmail.com> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Switch to CompletionStage Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG entry Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <andrebvandeven@gmail.com> Signed-off-by: Andre van de Ven <113951599+andrevandeven@users.noreply.github.com> Signed-off-by: Andre van de Ven <andrevdv@amazon.com> Co-authored-by: Andre van de Ven <andrevdv@amazon.com> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <anurag.rai@uber.com> Signed-off-by: Anurag Rai <91844619+anuragrai16@users.noreply.github.com> * Add Changelog entry Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Add secondary constructor Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Modify changelog Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Update changelog Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <gptrajat@amazon.com> --------- Signed-off-by: Rajat Gupta <gptrajat@amazon.com> Signed-off-by: Atri Sharma <atri.jiit@gmail.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Andre van de Ven <andrebvandeven@gmail.com> Signed-off-by: Andre van de Ven <113951599+andrevandeven@users.noreply.github.com> Signed-off-by: Andre van de Ven <andrevdv@amazon.com> Signed-off-by: Anurag Rai <anurag.rai@uber.com> Signed-off-by: Anurag Rai <91844619+anuragrai16@users.noreply.github.com> Co-authored-by: Rajat Gupta <gptrajat@amazon.com> Co-authored-by: Atri Sharma <atri.jiit@gmail.com> Co-authored-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Andre van de Ven <113951599+andrevandeven@users.noreply.github.com> Co-authored-by: Andre van de Ven <andrevdv@amazon.com> Co-authored-by: Anurag Rai <91844619+anuragrai16@users.noreply.github.com> Signed-off-by: Ankit Jain <jainankitk@apache.org>

* Add overload for channelFactory Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Fix tests Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Add Changelog entry Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Fix conflicts Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * When update operations fail during preparation (e.g., version conflicts), (opensearch-project#18917) TransportShardBulkAction still triggers refresh even though no actual writes occurred. This fix checks if locationToSync is null (indicating no writes) and prevents refresh in such cases. Fixes opensearch-project#15261 Signed-off-by: Atri Sharma <atri.jiit@gmail.com> * Remove all entries from changelog to be released in 3.2 (opensearch-project#18989) Signed-off-by: Andrew Ross <andrross@amazon.com> * Add temporal routing processors for time-based document routing (opensearch-project#18966) Implements TemporalRoutingProcessor for ingest pipelines and TemporalRoutingSearchProcessor for search pipelines based on RFC opensearch-project#18920. Features: - Route documents to shards based on timestamp fields - Support hour, day, week, and month granularities - Optional hash bucketing for better distribution - Automatic search routing to relevant time ranges - ISO week format support The processors enable efficient time-based data organization for log and metrics workloads by co-locating documents from the same time period on the same shards. --------- Signed-off-by: Atri Sharma <atri.jiit@gmail.com> * Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (opensearch-project#18998) * Add CompletableFuture variables to methods in the Client Interface and default to ActionListener impl Signed-off-by: Craig Perkins <cwperx@amazon.com> * Add to CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Fix typo in CHANGELOG Signed-off-by: Craig Perkins <cwperx@amazon.com> * Switch to CompletionStage Signed-off-by: Craig Perkins <cwperx@amazon.com> * Update CHANGELOG entry Signed-off-by: Craig Perkins <cwperx@amazon.com> --------- Signed-off-by: Craig Perkins <cwperx@amazon.com> * Expand fetch phase profiling to support inner hits and top hits aggregation phases (opensearch-project#18936) --------- Signed-off-by: Andre van de Ven <andrebvandeven@gmail.com> Signed-off-by: Andre van de Ven <113951599+andrevandeven@users.noreply.github.com> Signed-off-by: Andre van de Ven <andrevdv@amazon.com> Co-authored-by: Andre van de Ven <andrevdv@amazon.com> * IllegalArgumentException when scroll ID has a node no longer part of the Cluster (opensearch-project#19031) --------- Signed-off-by: Anurag Rai <anurag.rai@uber.com> Signed-off-by: Anurag Rai <91844619+anuragrai16@users.noreply.github.com> * Add Changelog entry Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Add secondary constructor Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Modify changelog Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Update changelog Signed-off-by: Rajat Gupta <gptrajat@amazon.com> * Add another constructor to fix breaking change check Signed-off-by: Rajat Gupta <gptrajat@amazon.com> --------- Signed-off-by: Rajat Gupta <gptrajat@amazon.com> Signed-off-by: Atri Sharma <atri.jiit@gmail.com> Signed-off-by: Andrew Ross <andrross@amazon.com> Signed-off-by: Craig Perkins <cwperx@amazon.com> Signed-off-by: Andre van de Ven <andrebvandeven@gmail.com> Signed-off-by: Andre van de Ven <113951599+andrevandeven@users.noreply.github.com> Signed-off-by: Andre van de Ven <andrevdv@amazon.com> Signed-off-by: Anurag Rai <anurag.rai@uber.com> Signed-off-by: Anurag Rai <91844619+anuragrai16@users.noreply.github.com> Co-authored-by: Rajat Gupta <gptrajat@amazon.com> Co-authored-by: Atri Sharma <atri.jiit@gmail.com> Co-authored-by: Andrew Ross <andrross@amazon.com> Co-authored-by: Craig Perkins <cwperx@amazon.com> Co-authored-by: Andre van de Ven <113951599+andrevandeven@users.noreply.github.com> Co-authored-by: Andre van de Ven <andrevdv@amazon.com> Co-authored-by: Anurag Rai <91844619+anuragrai16@users.noreply.github.com>

RajatGupta02 requested review from a team, Bukhtawar, CEHENKLE, Rishikesh1159, VachaShah, anasalkouz, andrross, ashking94, cwperks, dbwiddis, gbbafna, jed326, kotwanikunal, mch2, msfroh, owaiskazi19, reta, sachinpkale, saratvemulapalli, shwetathareja and sohami as code owners August 5, 2025 08:33

opensearch-ci-bot mentioned this pull request Aug 5, 2025

[AUTOCUT] Gradle Check Flaky Test Report for RemoteFsTimestampAwareTranslogTests #15818

Open

opensearch-ci-bot mentioned this pull request Aug 5, 2025

[AUTOCUT] Gradle Check Flaky Test Report for RemoteRestoreSnapshotIT #14324

Open

Rajat Gupta added 2 commits August 26, 2025 19:40

Add secondary constructor

bafa5de

Signed-off-by: Rajat Gupta <gptrajat@amazon.com>

Modify changelog

83b5bb5

Signed-off-by: Rajat Gupta <gptrajat@amazon.com>

RajatGupta02 force-pushed the translog_overload branch from 282df68 to 83b5bb5 Compare August 26, 2025 14:11

Update changelog

6a3eced

Signed-off-by: Rajat Gupta <gptrajat@amazon.com>

Merge branch 'main' into translog_overload

51b082c

Signed-off-by: Craig Perkins <cwperx@amazon.com>

cwperks reviewed Aug 26, 2025

View reviewed changes

server/src/main/java/org/opensearch/index/translog/Translog.java Show resolved Hide resolved

Add another constructor to fix breaking change check

4c8df69

Signed-off-by: Rajat Gupta <gptrajat@amazon.com>

opensearch-ci-bot mentioned this pull request Aug 26, 2025

[AUTOCUT] Gradle Check Flaky Test Report for SearchPhaseControllerTests #19094

Open

opensearch-ci-bot mentioned this pull request Aug 24, 2025

[AUTOCUT] Gradle Check Flaky Test Report for DecommissionControllerTests #15833

Open

cwperks approved these changes Aug 27, 2025

View reviewed changes

cwperks merged commit 981127d into opensearch-project:main Aug 27, 2025
31 of 34 checks passed

This was referenced Aug 27, 2025

[AUTOCUT] Gradle Check Flaky Test Report for ResourceAwareTasksTests #14293

Closed

[AUTOCUT] Gradle Check Flaky Test Report for StreamStringTermsAggregatorTests #18956

Closed

[AUTOCUT] Gradle Check Flaky Test Report for LocalTranslogTests #19168

Closed

RajatGupta02 mentioned this pull request Sep 15, 2025

Backport to 3.1 for channelFactory param in translog #19288

Closed

3 tasks

BrewTestBot mentioned this pull request Oct 15, 2025

opensearch 3.3.0 Homebrew/homebrew-core#249627

Merged

This was referenced Jan 10, 2026

docs: add translog channel factory report for v3.3.0 tkykenmt/opensearch-feature-explorer#1517

Merged

[feature] Translog tkykenmt/opensearch-feature-explorer#1392

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Channel Factory parameter to Translog#18918

Add Channel Factory parameter to Translog#18918
cwperks merged 16 commits intoopensearch-project:mainfrom
RajatGupta02:translog_overload

RajatGupta02 commented Aug 5, 2025

Uh oh!

RajatGupta02 commented Aug 5, 2025

Uh oh!

github-actions bot commented Aug 5, 2025

Uh oh!

github-actions bot commented Aug 5, 2025

Uh oh!

cwperks commented Aug 6, 2025

Uh oh!

github-actions bot commented Aug 6, 2025

Uh oh!

codecov bot commented Aug 6, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Aug 6, 2025

Uh oh!

github-actions bot commented Aug 26, 2025

Uh oh!

github-actions bot commented Aug 26, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 26, 2025

Uh oh!

github-actions bot commented Aug 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

RajatGupta02 commented Aug 5, 2025

Description

Related Issues

Check List

Uh oh!

RajatGupta02 commented Aug 5, 2025

Uh oh!

github-actions bot commented Aug 5, 2025

Uh oh!

github-actions bot commented Aug 5, 2025

Uh oh!

cwperks commented Aug 6, 2025

Uh oh!

github-actions bot commented Aug 6, 2025

Uh oh!

codecov bot commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

github-actions bot commented Aug 6, 2025

Uh oh!

github-actions bot commented Aug 26, 2025

Uh oh!

github-actions bot commented Aug 26, 2025

Uh oh!

Uh oh!

github-actions bot commented Aug 26, 2025

Uh oh!

github-actions bot commented Aug 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

codecov bot commented Aug 6, 2025 •

edited

Loading