Check field storage when synthetic source is enabled, in tests#139715
Merged
Kubik42 merged 1 commit intoelastic:mainfrom Dec 18, 2025
Merged
Check field storage when synthetic source is enabled, in tests#139715Kubik42 merged 1 commit intoelastic:mainfrom
Kubik42 merged 1 commit intoelastic:mainfrom
Conversation
e521d15 to
0a947b2
Compare
0a947b2 to
da9e904
Compare
Collaborator
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
martijnvg
approved these changes
Dec 18, 2025
Member
martijnvg
left a comment
There was a problem hiding this comment.
LGTM, thx for including this in our test coverage!
szybia
added a commit
to szybia/elasticsearch
that referenced
this pull request
Dec 19, 2025
* upstream/main: (253 commits) Adds ST_SIMPLIFY geo spatial function (elastic#136309) Take control of max clause count verification in Lucene searcher (elastic#139752) [ML] Unmute Inference Test (elastic#139765) Parameterize the vector operation benchmark tests (elastic#139735) Fix node reduction pushdown tests for release tests (elastic#139548) Fix flakiness in TSDataGenerationHelper (elastic#139759) CPS: Copy existing resolved index expressions when constructing a new `SearchRequest` from an existing one (elastic#139596) Add release notes for v9.1.9 release (elastic#139674) Add lucene query for wildcards on high cardinality keyword fields. (elastic#139746) Suppress Tika entitlement warnings from AWT (elastic#139711) Check field storage when synthetic source is enabled, in tests (elastic#139715) Refactor VectorSimilarityType to know about its corresponding Function (elastic#139678) Merge fixes from patch branch back into main (elastic#139721) Define native bulk operations for vector square distance (elastic#139198) Use LongUpDownCounter for Linked Project Error Metrics (elastic#139657) ESQL: Add javadoc that explains version-aware planning (elastic#139706) Add helper to pick node for reindex relocation (elastic#139081) Fix auth serialization randomized version test (elastic#139182) ES|QL - Add parsing, preanalysis and analysis timing information to profile (elastic#139540) Mute org.elasticsearch.persistent.ClusterPersistentTasksCustomMetadataTests testMinVersionSerialization elastic#139741 ...
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently, when testing field mappers against synthetic source, we rarely check if fields are stored appropriately and not double stored. This has led to regressions in releases and in serverless that go unnoticed; the most recent example being #139415.
FieldStorageVerifieraims to help with that by providing a simple API that can be leveraged to verify that a field is stored exactly where we expect it to. The class itself looks for all instances of a given field in a given document. If said instances don't match expectations, it complains.Example use case:
Will verify that
nameis only stored in doc_values. Ifnameis stored anywhere else, like ignored_source or aStoredField, then the verification check will fail. Furthermore, ifnameis stored twice in doc_values, as with fallback fields, the check will also fail. This helps us confirm:nameis not doubled storednameis stored in the expected place; in this case doc_valuesI've updated
TextFieldMapperTestsfor now, with more field mappers to follow in future PRs.This closes #139550