Skip to content

Make AbstractDownsampleFieldProducer generic to faciltate more types#139033

Merged
elasticsearchmachine merged 6 commits intoelastic:mainfrom
gmarouli:refactor-downsample-field-producers
Dec 5, 2025
Merged

Make AbstractDownsampleFieldProducer generic to faciltate more types#139033
elasticsearchmachine merged 6 commits intoelastic:mainfrom
gmarouli:refactor-downsample-field-producers

Conversation

@gmarouli
Copy link
Copy Markdown
Contributor

@gmarouli gmarouli commented Dec 4, 2025

In #138561 we added exponential histograms and currently we are working on adding legacy histograms too. Every time we add a new doc value reader we need to add new types of field producer and doc value reader pairs. This PR makes that easier by making AbstractDownsampleFieldProducer generic.

This is extracted from the work to downsample legacy histograms to facilitate reviewing.

@gmarouli gmarouli added >refactoring :StorageEngine/Downsampling Downsampling (replacement for rollups) - Turn fine-grained time-based data into coarser-grained data labels Dec 4, 2025
@elasticsearchmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

Copy link
Copy Markdown
Contributor

@romseygeek romseygeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

final LeafDownsampleCollector.FieldCollector<?>[] fieldCollectors = new LeafDownsampleCollector.FieldCollector<
?>[fieldValueFetchers.size()];
for (int i = 0; i < fieldValueFetchers.size(); i++) {
var fieldValueFetcher = fieldValueFetchers.get(i);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe for a follow-up, but would this be easier to reason about if FieldValueFetcher had a fieldCollector() method so each implementation could build a FieldCollector with the appropriate typed iterator sources?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me give it some thought before I merge :).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took a look and it can work really well. The main idea is to make the FieldValueFetcher also generic use that in most places so that we do not need to dance between the two.

I would like to wait for the re-introduction of type specific last value producers because then it's going to work even better. Considering this a step to the right direction I will merge this. I will keep you posted.

@gmarouli gmarouli added the auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) label Dec 4, 2025
@elasticsearchmachine elasticsearchmachine merged commit 72e4dbb into elastic:main Dec 5, 2025
34 checks passed
@gmarouli gmarouli deleted the refactor-downsample-field-producers branch December 5, 2025 12:10
mamazzol pushed a commit to mamazzol/elasticsearch that referenced this pull request Dec 5, 2025
elastic#139033)

In elastic#138561 we added
exponential histograms and currently we are working on adding legacy
histograms too. Every time we add a new doc value reader we need to add
new types of field producer and doc value reader pairs. This PR makes
that easier by making `AbstractDownsampleFieldProducer` generic.

This is extracted from the work to downsample legacy histograms to
facilitate reviewing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) >refactoring :StorageEngine/Downsampling Downsampling (replacement for rollups) - Turn fine-grained time-based data into coarser-grained data Team:StorageEngine v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants