Allow shrinking of indices from a previous major#28076
Merged
ywelsch merged 1 commit intoelastic:masterfrom Jan 4, 2018
Merged
Allow shrinking of indices from a previous major#28076ywelsch merged 1 commit intoelastic:masterfrom
ywelsch merged 1 commit intoelastic:masterfrom
Conversation
s1monw
suggested changes
Jan 4, 2018
Contributor
s1monw
left a comment
There was a problem hiding this comment.
I left one suggestion looks good otherwise
| Lucene.cleanLuceneIndex(target); | ||
| assert sources.length > 0; | ||
| final int luceneIndexCreatedVersionMajor = Lucene.readSegmentInfos(sources[0]).getIndexCreatedVersionMajor(); | ||
| new SegmentInfos(luceneIndexCreatedVersionMajor).commit(target); |
Contributor
There was a problem hiding this comment.
Can we copy the first index into the target directory instead of writing a dummy commit point? something like this:
Directory dir = sources[0];
for (String file : dir.listAll()) {
target.copyFrom(dir, file, file, IOContext.DEFAULT);
}This way we also apply the same optimizations for hardlinking etc and we are totally safe?
Contributor
There was a problem hiding this comment.
Yannick and I discussed this option first, but this needs extra care, for instance to not copy the write lock. It's also a bit more involved if we want to track statistics as well for the first source. In the end it's not clear to me which option is better.
Contributor
Author
There was a problem hiding this comment.
Contributor
There was a problem hiding this comment.
I was just opening the issue but I'll wait to see the conclusion here first in case we decide copying the first directory manually is still a better trade-off.
s1monw
approved these changes
Jan 4, 2018
jpountz
approved these changes
Jan 4, 2018
Contributor
jpountz
left a comment
There was a problem hiding this comment.
LGTM! Thanks for tackling it.
ywelsch
added a commit
that referenced
this pull request
Jan 4, 2018
Lucene does not allow adding Lucene 6 files to a Lucene 7 index. This commit ensures that we carry over the Lucene version to the newly created Lucene index. Closes #28061
ywelsch
added a commit
that referenced
this pull request
Jan 4, 2018
Lucene does not allow adding Lucene 6 files to a Lucene 7 index. This commit ensures that we carry over the Lucene version to the newly created Lucene index. Closes #28061
martijnvg
added a commit
that referenced
this pull request
Jan 4, 2018
* es/master: (53 commits) Bump compat version for local depdendent test to 6.2.0 Pass `java.locale.providers=COMPAT` to Java 9 onwards (#28080) Allow shrinking of indices from a previous major (#28076) Remove deprecated exceptions (#28059) Add Writeable.Reader support to TransportResponseHandler (#28010) Plugins: Add plugin extension capabilities (#27881) Fix cluster.routing.allocation.enable and cluster.routing.rebalance.enable casing (#28037) [Test] Fix scores for dcg in RankEvalRequestIT and RankEvalYamlIT [Docs] Add note on limitation for significant_text with nested objects (#28052) Enable convert processor to support Long and Double. (#27957) Enable Wildfly tests on JDK 9 and JDK 10 [Test] Fix allowed delta for calculated scores in DiscountedCumulativeGainTests [Test] Mute DiscountedCumulativeGainTests on ARM Only bind loopback addresses when binding to local Fix assertion in Wildfly build Fix typo in comment in Wildfly build Use ephemeral ports in Wildfly tests Update fuzzy-query.asciidoc (#28032) Just another elasticsearch library (#27996) Disable nio test transport (#28028) ...
martijnvg
added a commit
that referenced
this pull request
Jan 4, 2018
* es/6.x: (48 commits) Bump compat version for local depdendent test to 6.2.0 Pass `java.locale.providers=COMPAT` to Java 9 onwards (#28080) Allow shrinking of indices from a previous major (#28076) Add Writeable.Reader support to TransportResponseHandler (#28010) Fix cluster.routing.allocation.enable and cluster.routing.rebalance.enable casing (#28037) [Test] Fix scores for dcg in RankEvalRequestIT and RankEvalYamlIT [Docs] Add note on limitation for significant_text with nested objects (#28052) [Test] Fix allowed delta for calculated scores in DiscountedCumulativeGainTests Enable convert processor to support Long and Double. (#27957) Enable Wildfly tests on JDK 9 and JDK 10 update ingest-attachment to use Tika 1.17 and newer deps (#27824) Only bind loopback addresses when binding to local Fix assertion in Wildfly build Fix typo in comment in Wildfly build Use ephemeral ports in Wildfly tests Update fuzzy-query.asciidoc (#28032) Add node id to shard failure message (#28024) Introduce limit to the number of terms in Terms Query (#27968) Upgrade Gradle Shadow plugin to 2.0.2 Upgrade to JMH 1.19 ...
jasontedor
added a commit
that referenced
this pull request
Jan 4, 2018
* master: Set the elasticsearch-nio codebase for tests (#28067) Bump compat version for local depdendent test to 6.2.0 Pass `java.locale.providers=COMPAT` to Java 9 onwards (#28080) Allow shrinking of indices from a previous major (#28076) Remove deprecated exceptions (#28059) Add Writeable.Reader support to TransportResponseHandler (#28010) Plugins: Add plugin extension capabilities (#27881)
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Jan 6, 2018
* master: (25 commits) Remove Gradle cheatsheet Fix reproduction info to point to Gradle wrapper Update platforms tests to use Gradle wrapper Update testing docs to reflect Gradle wrapper Painless: Modify Loader to Load Classes Directly from Definition (elastic#28088) Update contributing docs to use the Gradle wrapper Create nio-transport plugin for NioTransport (elastic#27949) test: replaced try-catch statements with expectThrows(...) Add getWarmer and getTranslog method to NodeIndicesStats (elastic#28092) fix doc mistake Added ASN support for Ingest GeoIP plugin. Fix global aggregation that requires breadth first and scores (elastic#27942) Introduce Gradle wrapper Ignore GIT_COMMIT when calculating commit hash Re-enable bwc tests after elastic#27881 was backported Set the elasticsearch-nio codebase for tests (elastic#28067) Bump compat version for local depdendent test to 6.2.0 Pass `java.locale.providers=COMPAT` to Java 9 onwards (elastic#28080) Allow shrinking of indices from a previous major (elastic#28076) Remove deprecated exceptions (elastic#28059) ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lucene does not allow adding Lucene 6 files to a Lucene 7 index. This PR ensures that we carry over the Lucene version to the newly created Lucene index.
Closes #28061